HydrogenAudio

Hydrogenaudio Forum => Validated News => Topic started by: Sebastian Mares on 2005-12-05 15:17:07

Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 15:17:07
Hello!

I would like to announce the launch of my public, multiformat listening test @ 128 kbps.

The encoders featured are:For further information on how to participate, please visit the test announcement page:
http://www.maresweb.de/listening-tests/mf_128_1.php (http://www.maresweb.de/listening-tests/mf_128_1.php)

The test is scheduled to end on December 25th, 2005.

Update: The test was extended to end on January 13th, 2006.

Thank you!

Regards,
Sebastian Mares
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 15:19:30
For those using BitTorrent, statistics are available here:

http://www.rarewares.org/tracker/mystats.php (http://www.rarewares.org/tracker/mystats.php)

Please use BT whenever possible.

Thanks to Roberto for sharing.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ff123 on 2005-12-05 15:39:13
Can you reformat the readme.txt file so that it wraps properly?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 15:40:02
Quote
Can you reformat the readme.txt file so that it wraps properly?
[a href="index.php?act=findpost&pid=347843"][{POST_SNAPBACK}][/a]


How many characters per line - 72?

I tried to wrap it around 70 characters. It's a bit more sometimes due to URLs.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: schnofler on 2005-12-05 16:24:29
Just a little note: If you can't hear sound when you press the Play button, try changing the playback device in Options->Settings->Playback. Seems like with some soundcards the default setting won't work.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ShowsOn on 2005-12-05 16:27:43
Quote
Just a little note: If you can't hear sound when you press the Play button, try changing the playback device in Options->Settings->Playback. Seems like with some soundcards the default setting won't work.
[a href="index.php?act=findpost&pid=347848"][{POST_SNAPBACK}][/a]


Good advice. For me it defaulted to my capture card, but it is easy to simply change it to primary sound driver.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-05 16:36:19
aacenc32.dll  ver. 3.1.0.2.  Nero7 has 3.0.0.2.  Is it newer aacenc than from Nero7 package?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 16:40:20
Quote
aacenc32.dll  ver. 3.1.0.2.  Nero7 has 3.0.0.2.  Is it newer aacenc than from Nero7 package?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347851")


Yes.

If you click on "Nero AAC 3.1.0.2" on the presentation page, it will let you download a ZIP containing the encoder DLLs. Also, check out [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=347563]this[/url] post.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-05 16:42:48
I made a html version of the readme.txt file. I fixed a couple of typos too.

readme.htm (http://kotisivu.mtv3.fi/alexb/ha/readme.htm)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 16:44:18
Quote
I made a html version of the readme.txt file. I fixed a couple of typos too.

readme.htm (http://kotisivu.mtv3.fi/alexb/ha/readme.htm)
[a href="index.php?act=findpost&pid=347855"][{POST_SNAPBACK}][/a]


Ah, nice, thanks. What typos?  And do you mind if I update the ABC-HR_bin.zip file to include your HTML?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-05 16:49:20
Quote
If you click on "Nero AAC 3.1.0.2" on the presentation page, it will let you download a ZIP containing the encoder DLLs. Also, check out this (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=347563) post.
[a href="index.php?act=findpost&pid=347853"][{POST_SNAPBACK}][/a]


Thanks. I´ve already noticed that.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-05 16:49:20
Use it as you like. Be my guest.

Typos (if i remember correctly):
then you in the beginning and extra spaces in the torrent links before the numbers 17 and 18.

Edit: typo 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Gambit on 2005-12-05 16:54:40
Conduct, conduct, conduct...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 16:54:48
Quote
Use it as you like. Be my guest.

Typos (if i remember correctly):
then you in the beginning and extra spaces in the torrent links before the numbers 17 and 18.

Edit: typo  
[a href="index.php?act=findpost&pid=347858"][{POST_SNAPBACK}][/a]


Ah thanks. Don't know what you mean with the torrent links, though.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-05 17:00:40
They were like this after I pasted the text file into Dreamweaver:

Sample 17.zip.torrent  TheDraperyFalls
Sample 18.zip.torrent  WhiteAmerica

Perhaps it was something on my side...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-05 17:41:51
Sebastian,

The TorrentStorm link in my html is wrong. It has extra html code included and doesn't work. I'll fix it in 5 minutes.


EDIT

I fixed it, but when I tested the link I found this:

Quote
News
30th March 2005.

For personal reasons, I have been unable to continue with the development of Torrentstorm, and have decided that until I can continue with its development, it is best to close this site. Thank you to all who have helped and supported torrentstorms development, and I hope to see you all again soon.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 17:45:40
Quote
Sebastian,

The Torrent Storm link in my html is wrong. It has extra html code included and doesn't work. I'll fix it in 5 minutes.
[a href="index.php?act=findpost&pid=347866"][{POST_SNAPBACK}][/a]


I re-did the HTML using plain HTML 4. No styles, whatsoever.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-05 18:01:24
I have currently 18 BitTornado (http://www.bittornado.com/) clients running. It seems to work, but my web browsing is very sluggish.

Personally I would prefer eMule (and the eD2K & Kad networks it uses).
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Gambit on 2005-12-05 18:05:10
Quote
I have currently 18 BitTornado (http://www.bittornado.com/) clients running. It seems to work, but my web browsing is very sluggish.

Personally I would prefer eMule (and the eD2K & Kad networks it uses).
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347871")

You just need a proper bittorrent client, like:
[a href="http://azureus.sourceforge.net/]Azureus[/url]
BitComet (http://www.bitcomet.com/index.htm)
µTorrent (http://www.utorrent.com/)

Oh, and it still says "conduce" everywhere on the pages...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: eisa01 on 2005-12-05 18:06:53
Hey, I'm wondering if I should try and participate in this listening test. I have never done anything like this, but I think it should be interesting. Now, in the readme, it says that headphones are a must. I have a pair of Bose QuietComfort, that have noise cancellation. Would they be usable?

And just some advice regarding the bittorrent stuff. You should have put all the sample archives in one torrent file, then it's much easier to seed/leech. For those of you complaining of slow browsing, when you're running bittorrent, you need to limit your upload speed in the client. Now, a good client for windows, that have one window for several torrents, is µTorrent (http://www.utorrent.com).
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 18:16:41
Quote
Oh, and it still says "conduce" everywhere on the pages...
[a href="index.php?act=findpost&pid=347873"][{POST_SNAPBACK}][/a]


Ah! Sorry, didn't get that the first time you posted.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 18:18:30
Quote
Hey, I'm wondering if I should try and participate in this listening test. I have never done anything like this, but I think it should be interesting. Now, in the readme, it says that headphones are a must. I have a pair of Bose QuietComfort, that have noise cancellation. Would they be usable?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347874")


I think so, I don't have much experience with headphones.

Quote
And just some advice regarding the bittorrent stuff. You should have put all the sample archives in one torrent file, then it's much easier to seed/leech. For those of you complaining of slow browsing, when you're running bittorrent, you need to limit your upload speed in the client. Now, a good client for windows, that have one window for several torrents, is [a href="http://www.utorrent.com]µTorrent[/url].
[a href="index.php?act=findpost&pid=347874"][{POST_SNAPBACK}][/a]


I will tell that Roberto he's responsible for hosting the samples and preparing the torrents.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-05 18:45:34
Quote
I made a html version of the readme.txt file. I fixed a couple of typos too.

readme.htm (http://kotisivu.mtv3.fi/alexb/ha/readme.htm)
[a href="index.php?act=findpost&pid=347855"][{POST_SNAPBACK}][/a]


I changed the file again. It has the three mentioned client links.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-05 18:55:18
Hm ... Why nobody care about final size? It´s already normal.
¨It´s VBR. It´s normal that  real bitrate will be 140 kbit/s while nominal bitrate 128¨
I think it´s normal that audiocodec can produce some deviation from nominal bitrate . But there is difference between final size of files encoded by different  audiocodecs.  Ok , maybe itunes better than nero, but itunes almost always has a bigger file size. Maybe at the same file size the result will be different.

Yes, I´m caring about final size. My mp3player has 256 mb. And If my mp3s will have 264 mb I doubt I can explain point about VBR  to my mp3player.

The same discussion was about  testing of videocodecs. Some videocodecs always produced slightly  bigger size than others. But it´s  not reason to left it as is.  It´s impossible to feat 4600 mb gb to 1 standard DVD 4486 Mb.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Garf on 2005-12-05 18:56:37
If bittorrent is an issue, they can go as HTTP or FTP downloads on the HA server. Bandwidth shouldn't be an issue, we have enough left.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ep0ch on 2005-12-05 19:03:43
Why are there 18 torrents and not just one big one?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 19:04:50
Quote
Hm ... Why nobody care about final size? It´s already normal.
¨It´s VBR. It´s normal that  real bitrate will be 140 kbit/s while nominal bitrate 128¨
I think it´s normal that audiocodec can produce some deviation from nominal bitrate . But there is difference between final size of files encoded by different  audiocodecs.  Ok , maybe itunes better than nero, but itunes almost always has a bigger file size. Maybe at the same file size the result will be different.

Yes, I´m caring about final size. My mp3player has 256 mb. And If my mp3s will have 264 mb I doubt I can explain point about VBR  to my mp3player.

The same discussion was about  testing of videocodecs. Some videocodecs always produced slightly  bigger size than others. But it´s  not reason to left it as is.  It´s impossible to feat 4600 mb gb to 1 standard DVD 4486 Mb.
[a href="index.php?act=findpost&pid=347886"][{POST_SNAPBACK}][/a]


We are talking about a special case here. The settings used were chosen because they average to 128 kbps on a large set of audio tracks. What we test here are short and usually difficult to encode samples and that's why the bitrate goes up. And if you want predictable file size, you can use ABR for your MP3 player.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 19:06:07
Quote
Why are there 18 torrents and not just one big one?
[a href="index.php?act=findpost&pid=347889"][{POST_SNAPBACK}][/a]


Because some people might not want to download the whole sample package. I will ask Roberto to additionally create a torrent containing all samples and maybe also ZIP all samples to one file for HTTP DLs.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-05 19:12:56
Quote
And if you want predictable file size, you can use ABR for your MP3 player.
[a href="index.php?act=findpost&pid=347890"][{POST_SNAPBACK}][/a]


Thank you for advice but it wasn´t my point. 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: jetpower on 2005-12-05 19:17:15
Quote
Quote
Why are there 18 torrents and not just one big one?
[a href="index.php?act=findpost&pid=347889"][{POST_SNAPBACK}][/a]


Because some people might not want to download the whole sample package. I will ask Roberto to additionally create a torrent containing all samples and maybe also ZIP all samples to one file for HTTP DLs.
[a href="index.php?act=findpost&pid=347892"][{POST_SNAPBACK}][/a]


I think it will be more convenient to have one torrent with all files included seperately. AFAIK BitComet and Azureus allow users to select individual files from a torrent.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 19:33:27
OK, as I said, I am going to forward this to Roberto.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sTisTi on 2005-12-05 20:10:13
Oh damn, these codecs are really too good for me 
I've just gone through some samples, and while I can easily detect the low anchor, everything else sounds the same to me. All my ABX trials keep failing, this is really frustrating. I expected it to be pretty hard, but not this hard 
How are your experiences? can anyone suggest a good starter sample which is easier than the rest? I spent 20 minutes concentrating on LesJoursHeureux because I thought it might be the easiest, but no luck - it all sounds the same to me
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-05 20:27:47
Sample 2 has one of them with quite noticeable lowpass, and the low anchor which is an obvious tincan;
Elizabeth has a part where one of the noise backgrounds sounds half a step down, (eg lower frequencies), plus the tincan
Carbonelli has two samples that have the same effect as dr4 in http://ff123.net/training/training.html (http://ff123.net/training/training.html)

Otherwise, I agree -- I'm quite surprised by the quality of modern codecs and the amount of "5s" that are in them. -- I tried a "private" 64 and 96 kbps test, too, and failed (to a certain extent) -- It seems I don't have good artifact recognition after all.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-05 21:30:50
Oh my... This is not going to be easy... 

Think I'll just put them all at 5.0, send it all in and have it done with... 

No, but seriously, this is going to take ages... heh.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sTisTi on 2005-12-05 21:31:19
Quote
Sample 2 has one of them with quite noticeable lowpass, and the low anchor which is an obvious tincan;
Elizabeth has a part where one of the noise backgrounds sounds half a step down, (eg lower frequencies), plus the tincan
Carbonelli has two samples that have the same effect as dr4 in http://ff123.net/training/training.html (http://ff123.net/training/training.html)

Otherwise, I agree -- I'm quite surprised by the quality of modern codecs and the amount of "5s" that are in them. -- I tried a "private" 64 and 96 kbps test, too, and failed (to a certain extent) -- It seems I don't have good artifact recognition after all.
[a href="index.php?act=findpost&pid=347910"][{POST_SNAPBACK}][/a]

Thanks for the tip with Carbonelli, I managed successful ABX results with that one finally  It's a good sample because you can concentrate more easily one one thing at a time. Most other samples are pretty chaotic sounding, which makes it hard to focus one's attention.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-05 22:06:29
Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure... 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 22:07:59
Quote
Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure...  
[a href="index.php?act=findpost&pid=347928"][{POST_SNAPBACK}][/a]


It should start playing at 2 seconds. That is not a bug, it's a feature.

Quote
[...]let encoders adapt themselves to the content by not testing the first few seconds of the sample.
If you encode a few seconds at the beginning that are not tested by the listener, you solve the problems of psymodel adaptation and bitrate management adaptation.
[a href="index.php?act=findpost&pid=346598"][{POST_SNAPBACK}][/a]
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-05 22:13:36
Quote
Quote
Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure...  
[a href="index.php?act=findpost&pid=347928"][{POST_SNAPBACK}][/a]


It should start playing at 2 seconds. That is not a bug, it's a feature.
[a href="index.php?act=findpost&pid=347929"][{POST_SNAPBACK}][/a]


What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 22:23:31
Quote
Quote
Quote
Oh, btw - ABC/HR seems to start playing the samples at about .3 secs every so often. Is this a known issue? It's really annoying, that's for sure...  
[a href="index.php?act=findpost&pid=347928"][{POST_SNAPBACK}][/a]


It should start playing at 2 seconds. That is not a bug, it's a feature.
[a href="index.php?act=findpost&pid=347929"][{POST_SNAPBACK}][/a]


What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.
[a href="index.php?act=findpost&pid=347932"][{POST_SNAPBACK}][/a]


No idea, maybe schnofler can say something about this. I told ABC/HR to calculate offsets, so that shouldn't be a problem.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: schnofler on 2005-12-05 22:24:10
Quote
What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.
[a href="index.php?act=findpost&pid=347932"][{POST_SNAPBACK}][/a]

Do you use the Java Sound Engine or a native output device?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-05 22:30:50
Here's the torrent with all samples:
http://www.rarewares.org/torrents/Samples.torrent (http://www.rarewares.org/torrents/Samples.torrent)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-05 22:31:51
Quote
Quote
What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.
[a href="index.php?act=findpost&pid=347932"][{POST_SNAPBACK}][/a]

Do you use the Java Sound Engine or a native output device?
[a href="index.php?act=findpost&pid=347938"][{POST_SNAPBACK}][/a]


Ah - thanks! (Did I miss that recommendation somewhere?)

Can't seem to reproduce it when I use Java Sound Engine.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-05 22:33:46
Quote
Quote
Quote
What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.
[a href="index.php?act=findpost&pid=347932"][{POST_SNAPBACK}][/a]

Do you use the Java Sound Engine or a native output device?
[a href="index.php?act=findpost&pid=347938"][{POST_SNAPBACK}][/a]


Ah - thanks! (Did I miss that recommendation somewhere?)

Can't seem to reproduce it when I use Java Sound Engine.
[a href="index.php?act=findpost&pid=347941"][{POST_SNAPBACK}][/a]


Shouldn't it be the other way round? I mean, better result when using a native device.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-05 22:35:31
Quote
Shouldn't it be the other way round? I mean, better result when using a native device.
[a href="index.php?act=findpost&pid=347942"][{POST_SNAPBACK}][/a]


My thought exactly 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: schnofler on 2005-12-05 22:42:21
Quote
Quote
Quote
What I'm talking about happens even if you select a small part in the middle of the sample. While ABX-ing, if you switch quickly between the tracks by pressing 'q', 'w' and 'e' - quite often (but not always) the selection won't be played from the very start.
[a href="index.php?act=findpost&pid=347932"][{POST_SNAPBACK}][/a]

Do you use the Java Sound Engine or a native output device?
[a href="index.php?act=findpost&pid=347938"][{POST_SNAPBACK}][/a]


Ah - thanks! (Did I miss that recommendation somewhere?)

Can't seem to reproduce it when I use Java Sound Engine.
[a href="index.php?act=findpost&pid=347941"][{POST_SNAPBACK}][/a]

That wasn't a recommendation. Actually I'd recommend against using the Java Sound Engine, since, well, it sucks. Anyway, the problems you describe shouldn't occur either when using native output. Maybe twiddling with the buffer size will help. Otherwise, could you please describe in more detail how to reproduce these problems? Thanks.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-05 22:56:09
Quote
That wasn't a recommendation. Actually I'd recommend against using the Java Sound Engine, since, well, it sucks. Anyway, the problems you describe shouldn't occur either when using native output. Maybe twiddling with the buffer size will help. Otherwise, could you please describe in more detail how to reproduce these problems? Thanks.
[a href="index.php?act=findpost&pid=347945"][{POST_SNAPBACK}][/a]


Ok.

I tried changing the buffer size, and it seems to directly affect how far in playback will start when switching between samples. More details in a PM.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: skelly831 on 2005-12-06 00:59:33
For a 128kbps test, this if f'ing hard!
The results should be quite impressive
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-06 01:15:02
Quote
For a 128kbps test, this if f'ing hard!
The results should be quite impressive [a href="index.php?act=findpost&pid=347972"][{POST_SNAPBACK}][/a]


Those sentences are almost contradictory :B
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: skelly831 on 2005-12-06 01:27:54
Quote
Quote
For a 128kbps test, this if f'ing hard!
The results should be quite impressive [a href="index.php?act=findpost&pid=347972"][{POST_SNAPBACK}][/a]


Those sentences are almost contradictory :B
[a href="index.php?act=findpost&pid=347975"][{POST_SNAPBACK}][/a]

LOL!, you're right, maybe you can tell English isn't my native language, I meant to say the results may not be that impressive, but we'll all be a little wiser in the matter.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-06 01:52:50
Quote
LOL!, you're right, maybe you can tell English isn't my native language, I meant to say the results may not be that impressive, but we'll all be a little wiser in the matter.[a href="index.php?act=findpost&pid=347977"][{POST_SNAPBACK}][/a]


No, I mean, they are contradictory in a practical point of view, not a semantical one.

If this test really is "f'ing hard", people will be scared by it and result submittal rates will be low. Or they will manage to test the codecs anyway but rank several encoders near the top, tying everything together. In either case, results will be less than impressive :/


I have already placed my bets on who will win, but such comments are unwelcome until the test ends, so I'll keep them to myself
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: skelly831 on 2005-12-06 01:56:05
Quote
I have already placed my bets on who will win, but such comments are unwelcome until the test ends, so I'll keep them to myself
[a href="index.php?act=findpost&pid=347981"][{POST_SNAPBACK}][/a]

The above should be recommended to all who take the test, somebody always comes and ruins the uh... fun.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: jido on 2005-12-06 02:23:09
Ok, so who wrote a *nix script to decompress all the samples automatically?

C'mon shouldn't be difficult...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ronnie_t on 2005-12-06 03:43:11
I will be hosting the files on razorback2
Code: [Select]
ed2k://|file|Sample01.zip|8509511|66679758AA8A2A35A1E84A689A1ECD53|h=LWZMHFG5CETFRHK6WLGEJPIXDQP4EBL6|/
ed2k://|file|Sample02.zip|7640515|AC5CDCBB42A2BD07EF814ABC32B48AFF|h=O2MEA225C5HB4I6JHAZXRUYDJTNM22CO|/
ed2k://|file|Sample03.zip|4628261|9F030183C15114A63E99AD1974B15BC9|h=SP3J4IZK7HPPYQUDI3DCYBJIGNTXC4XH|/
ed2k://|file|Sample04.zip|7034692|F793E41485B4B344306A6F3D3807529E|h=R3FA2SO5S725S2W2TIL7QOAQOVAVYR5K|/
ed2k://|file|Sample05.zip|8916801|86CA6334E2C822671BAA95E0F7DFFD8A|h=4AFNVZAADWZ6BMYY2GBWXKUP7KSRRJP2|/
ed2k://|file|Sample06.zip|3292659|82AA542444E3222E018EF0D1C65EF1D3|h=UMXYXWBJ57G45EFNTN3KA5QWSCFKJK6V|/
ed2k://|file|Sample07.zip|5947370|2C3A582CD5B0AB6BD8B0BDD21DDE2764|h=P7GV6IJG7TDUTN2OXE7BW2YJOZC4HDF5|/
ed2k://|file|Sample08.zip|7929888|BA0764358B80A8A67DDD00F385B15844|h=4E4BSA6X66LXAX5V7IQY7H2DS4TXCC3Z|/
ed2k://|file|Sample09.zip|8757039|D88B3A64A4C029FC250FA0A12C266906|h=JH2UTISCN5FDVXR4TKLTIP3IQZ2TBLIA|/
ed2k://|file|Sample10.zip|6620142|7F74993053571111DD9CFD38E9E31D98|h=355SD4TZTSYWZ6VIYGP5DNJKZXJERS4P|/
ed2k://|file|Sample11.zip|5358761|D0FAD27E576091EA0207B71BC47BD502|h=LGPPNNNCR7R2DQOQLGZTHJ44MVIB3XSW|/
ed2k://|file|Sample12.zip|9849316|DDBF5E2F93A0835DC197CFB4471957F0|h=PZDEHZT334ASEH7GBZSGSDB3RNHTK37D|/
ed2k://|file|Sample13.zip|8844385|A2F0D44C5ECCD1C00C92298FF902C7EA|h=ARSPD4C2B2YAKJG4HGEKSCNB33WMMRIM|/
ed2k://|file|Sample14.zip|6126723|6B18903D8DC559A9EE2523937A352D68|h=AVU5KZO7SHHKUHQRB3LNRE3LMRZXB23Z|/
ed2k://|file|Sample15.zip|5303587|8590704FDB32DC9074DBF0F49E3D8763|h=FY56ERKEKGQ36F5IF4YKLXC5Y3GCRG2C|/
ed2k://|file|Sample16.zip|4895823|40CA8C35902CA1033338C7F11A1881FB|h=DGOL3XMWKJ7SEHP6T6LNBORP4OLWIHPX|/
ed2k://|file|Sample17.zip|10289090|B662CD9548D880C6A9165C9E3C4B640E|h=6CE6UXJ5KBKFSCVTMM5OAE3IM5IMGB7P|/
ed2k://|file|Sample18.zip|9455856|A7727AAF3DC1AD2B384DC6056B63712E|h=ERZQI6KELOFJYHNUUCJCYUXD67GDVRUV|/
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ErikS on 2005-12-06 04:52:38
Quote
Ok, so who wrote a *nix script to decompress all the samples automatically?

C'mon shouldn't be difficult...
[a href=\"index.php?act=findpost&pid=347990\"][{POST_SNAPBACK}][/a]

I did for myself and sent a copy to Sebastian. Feel free to use... Place it in a file .../ABC-HR/bin_nix/SampleNN

It works for me so don't complain on my script skillz, m'kay.

Code: [Select]
#!/bin/bash

pushd .
cdir=`pwd`
bin=`dirname "$cdir/$0"`
cd $bin/..
nr=`echo $0 | sed 's/.*Sample\([0-9]*\)/\1/'`
oggdec=$bin/oggdec
faad=$bin/faad2
lame=$bin/lame
wvunpack=$bin/wvunpack

if [ -f "Sample${nr}.zip" ]; then
unzip Sample${nr}.zip
pushd .
cd Sample${nr}
for i in *
do
case $i in
*.mp3)
$lame --decode $i `basename $i mp3`wav
;;
*.mp4)
$faad $i
;;
*.m4a)
$faad $i
;;
*.ogg)
$oggdec $i
;;
*.wv)
$wvunpack $i
;;
esac
done
#remove compressed files
rm *.ogg *.wv *.mp3 *.mp4 *.m4a
popd
else
echo "Please download Sample${nr}.zip first."
fi

popd
then this in another file in the same directory:
Code: [Select]
#!/bin/bash

cdir=`pwd`
bin=`dirname "$cdir/$0"`

${bin}/Sample01
${bin}/Sample02
${bin}/Sample03
${bin}/Sample04
${bin}/Sample05
${bin}/Sample06
${bin}/Sample07
${bin}/Sample08
${bin}/Sample09
${bin}/Sample10
${bin}/Sample11
${bin}/Sample12
${bin}/Sample13
${bin}/Sample14
${bin}/Sample15
${bin}/Sample16
${bin}/Sample17
${bin}/Sample18

[span style=\'font-size:8pt;line-height:100%\']edit: suggested changes made...[/span]
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: martythestickman on 2005-12-06 07:00:02
hey guys.

just finished setting up all the files needed for the test including installation of java, and i have a problem. the wav's dont play. what am i doing wrong?
reference file doesnt play, ABX option playback doesnt work either. checked settings and my sound card is selected.....
need help if you want some results

thanks all
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 08:13:00
Quote
...
[a href="index.php?act=findpost&pid=348002"][{POST_SNAPBACK}][/a]


Thanks, I'll add it to the ABC-HR_bin.zip when I am back home if you don't mind.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: caligae on 2005-12-06 08:19:35
Quote
Ok, so who wrote a *nix script to decompress all the samples automatically?

C'mon shouldn't be difficult...
[a href="index.php?act=findpost&pid=347990"][{POST_SNAPBACK}][/a]


Wrote this script yesterday. This does no unpacking, error checking and so on. Simple go to directory SampleXX and run the script. This assumes all binaries are in $PATH. I think it's a bit easier extensible for future listening tests than ErikSs script so maybe we could merge them.

Code: [Select]
#!/bin/sh

for i in *
do
   case $i in
       *.mp3)
           lame --decode $i `basename $i mp3`wav
          ;;
       *.mp4)
           faad -o ` basename $i mp4`wav $i
          ;;
       *.m4a)
           faad -o `basename $i m4a`wav $i
          ;;
       *.ogg)
           oggdec $i
          ;;
       *.wv)
           wvunpack $i `basename $i wv`wav
          ;;
   esac
done
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 08:35:13
Could someone announce the test on D9 and /. please?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kurtnoise on 2005-12-06 09:25:48
Quote
Could someone announce the test on D9 and /. please?
[a href="index.php?act=findpost&pid=348021"][{POST_SNAPBACK}][/a]

I did it for D9...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2005-12-06 12:25:01
whats D9?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-06 12:29:39
http://www.doom9.org/ (http://www.doom9.org/)
http://slashdot.org/ (http://slashdot.org/)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2005-12-06 12:52:25
i was think in of digg.com ... we should digg it
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: eisa01 on 2005-12-06 14:14:53
Quote
Quote
Ok, so who wrote a *nix script to decompress all the samples automatically?

C'mon shouldn't be difficult...
[a href="index.php?act=findpost&pid=347990"][{POST_SNAPBACK}][/a]

Wrote this script yesterday. This does no unpacking, error checking and so on. Simple go to directory SampleXX and run the script. This assumes all binaries are in $PATH. I think it's a bit easier extensible for future listening tests than ErikSs script so maybe we could merge them.
[a href="index.php?act=findpost&pid=348019"][{POST_SNAPBACK}][/a]

Thanks, I'm pretty new to commandline stuff, so I had no idea how to do this. But that didn't stop me from modifying this line
Code: [Select]
for i in Sample??/*

Now, you just place it in the directory containing the sample folders and execute it.

edit: Maybe not a very good idea, because all the files are in the top folder now
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-06 15:23:52
Quote
...
Carbonelli has two samples that have the same effect as dr4 in http://ff123.net/training/training.html (http://ff123.net/training/training.html)
[a href="index.php?act=findpost&pid=347910"][{POST_SNAPBACK}][/a]


I really can't find the artifacts you're talking about. Are they at one particular position in the samples?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Atreus on 2005-12-06 15:35:28
Digg it!  Let's try to get this test on the front page...

http://digg.com/music/Multiformat_Listenin...128_kbps_-_OPEN (http://digg.com/music/Multiformat_Listening_Test_@_128_kbps_-_OPEN)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-06 18:52:31
Quote
Quote
,Dec 5 2005, 10:27 PM]...
Carbonelli has two samples that have the same effect as dr4 in http://ff123.net/training/training.html (http://ff123.net/training/training.html)
[a href="index.php?act=findpost&pid=347910"][{POST_SNAPBACK}][/a]


I really can't find the artifacts you're talking about. Are they at one particular position in the samples?
[a href="index.php?act=findpost&pid=348081"][{POST_SNAPBACK}][/a]

Near the beginning, IIRC.  I had volume quite loud on my headphones (hd-280 pro)  It's subtle, but recognizable.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ep0ch on 2005-12-06 19:04:32
Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 19:07:11
It is.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-06 19:20:00
VBR can produce different size. It will be normal if codec A will produce bigger size than codec B on one half of samples and codec B will produce bigger size on another half of samples.

However in this test. Itunes AAC always produces bigger size than Nero AAC.

1. +10%
2. +7.7%
3. +4.6%
4. +15%
5. +12%
6. +17.3%
7. +1.76%
8. +11.85%
9. +13.3 %
10. +9%
11. +8.3%
12. +12.6%
13. +10.37%
14. +15%
15. +9.3%
16. +8.57%
17. +9.1%
18. +1.74%

Neither one Nero AAC sample has a higher bitrate than Itunes  one. 
Probability that it happens only on this 18 samples is quite low.
It's not hard to assume iTunes will be better on this test  due to quite big oversize.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sTisTi on 2005-12-06 19:37:42
Quote
Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?
[a href="index.php?act=findpost&pid=348120"][{POST_SNAPBACK}][/a]

The whole procedure of a listening test kind of defeats itself if you think like this. Training yourself to hear artifacts also shouldn't be done then, because you then might recognize new artifacts you missed before, which would change the notation of the codecs. I think the important thing is whether you can hear artifacts are not. Without knowing what to listen for and preferably also in which parts of the music, I am pretty sure practically everybody would have to rate all codecs as 5.0 in this test.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 19:44:50
Quote
Quote
Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?
[a href="index.php?act=findpost&pid=348120"][{POST_SNAPBACK}][/a]

The whole procedure of a listening test kind of defeats itself if you think like this. Training yourself to hear artifacts also shouldn't be done then, because you then might recognize new artifacts you missed before, which would change the notation of the codecs. I think the important thing is whether you can hear artifacts are not. Without knowing what to listen for and preferably also in which parts of the music, I am pretty sure practically everybody would have to rate all codecs as 5.0 in this test.
[a href="index.php?act=findpost&pid=348126"][{POST_SNAPBACK}][/a]


Well, specifying the exact position and sample where an artifact is noticable is a bit unfair IMHO.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 19:46:32
Quote
VBR can produce different size. It will be normal if codec A will produce bigger size than codec B on one half of samples and codec B will produce bigger size on another half of samples.

However in this test. Itunes AAC always produces bigger size than Nero AAC.

1. +10%
2. +7.7%
3. +4.6%
4. +15%
5. +12%
6. +17.3%
7. +1.76%
8. +11.85%
9. +13.3 %
10. +9%
11. +8.3%
12. +12.6%
13. +10.37%
14. +15%
15. +9.3%
16. +8.57%
17. +9.1%
18. +1.74%

Neither one Nero AAC sample has a higher bitrate than Itunes  one. 
Probability that it happens only on this 18 samples is quite low.
It's not hard to assume iTunes will be better on this test   due to quite big oversize.
[a href="index.php?act=findpost&pid=348124"][{POST_SNAPBACK}][/a]


And that's why only a difference of 10% is tolerated. If one encoder is smart and assigns more bits - well - it's smart.
Also high bitrate doesn't automatically mean good quality.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-06 19:52:09
Quote
And that's why only a difference of 10% is tolerated. If one encoder is smart and assigns more bits - well - it's smart.


In the last Guru's test  difference also was about 10% between result of Nero and Itunes.
So if Nero will changes their codec in the way to produce  bigger size it will be a new revolutional codec, won't it?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-06 19:58:43
The sample bitrates are completely irrelevant and should be ignored. The VBR settings used with the other encoders besides Nero were chosen because they are likely to produce the same average bitrate with a library of complete tracks. The encoders are free to use any bitrate for the samples. Actually, we have no idea about the peak bitrates inside the samples. The only thing what matters is quality. Different encoders use different means.

The Nero developers who are aware of this criterion instructed the settings for this new untested Nero encoder.

EDIT

The older Nero 7 encoder produced the same average bitrate with the other encoders when the "Internet" VBR setting was tested by guruboolez and myself.

EDIT 2

I am going to add the new Nero version to my bitrate table soon.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 20:34:39
Quote
So if Nero will changes their codec in the way to produce  bigger size it will be a new revolutional codec, won't it?
[a href="index.php?act=findpost&pid=348131"][{POST_SNAPBACK}][/a]


As I said, bitrate is not the only factor when it comes to audio quality. Shine at 128 kbps CBR has the same bitrate as LAME at 128 kbps CBR. They both produce MP3s, but the resulting files sound totally different.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-06 20:47:16
Quote
Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?
[a href="index.php?act=findpost&pid=348120"][{POST_SNAPBACK}][/a]


Well, I had listened at the beginning and I couldn't hear a thing, so it certainly didn't help me 

Ok. I'll stop asking.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-06 20:52:08
Quote
Quote
Quote
Doesn't telling everyone where the artifacts are kinda defeat the point of a listening test?
[a href="index.php?act=findpost&pid=348120"][{POST_SNAPBACK}][/a]

The whole procedure of a listening test kind of defeats itself if you think like this. Training yourself to hear artifacts also shouldn't be done then, because you then might recognize new artifacts you missed before, which would change the notation of the codecs. I think the important thing is whether you can hear artifacts are not. Without knowing what to listen for and preferably also in which parts of the music, I am pretty sure practically everybody would have to rate all codecs as 5.0 in this test.
[a href="index.php?act=findpost&pid=348126"][{POST_SNAPBACK}][/a]


Well, specifying the exact position and sample where an artifact is noticable is a bit unfair IMHO.
[a href="index.php?act=findpost&pid=348128"][{POST_SNAPBACK}][/a]


But I think sTisTi has a point all the same. If I can hear it, I can hear it. If I can't - as it turned out in this case - (I can't even single out the low anchor for Carbonelli...) then I can't. All the knowledge in the world probably couldn't help me find that artifact, and hence the sample to me is a 5.0.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 21:00:46
That is true, too.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-06 21:13:31
Quote
That is true, too.
[a href="index.php?act=findpost&pid=348147"][{POST_SNAPBACK}][/a]


And you have to remember that the people taking a test like this probably vary a lot when it comes to training and skill, so the results can never be seen as representative for say 'average listeners' or 'pedantic listeners' - it's most likely a mix of all kinds.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-06 22:05:16
Jesus Christ!

Discussing artifacts ist verboten. Please stop it at once, people!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 22:06:11
Edit: Roberto was faster.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-06 22:19:12
Quote
Quote
That is true, too.
[a href="index.php?act=findpost&pid=348147"][{POST_SNAPBACK}][/a]


And you have to remember that the people taking a test like this probably vary a lot when it comes to training and skill, so the results can never be seen as representative for say 'average listeners' or 'pedantic listeners' - it's most likely a mix of all kinds.
[a href="index.php?act=findpost&pid=348154"][{POST_SNAPBACK}][/a]


One thing I didn't consider - when person X says that sample Y has an artifact at postition 123, then person Z is going to keep testing and testing sample Y because he knows that it has an artifact.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: caligae on 2005-12-06 23:57:10
Quote
One thing I didn't consider - when person X says that sample Y has an artifact at postition 123, then person Z is going to keep testing and testing sample Y because he knows that it has an artifact.
[a href="index.php?act=findpost&pid=348168"][{POST_SNAPBACK}][/a]


But this would only be unfair if that specific artifact is only spotable with encoder A, but encoder B has noticable artifacts at different positions. If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-07 00:48:18
Quote
If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.[a href="index.php?act=findpost&pid=348183"][{POST_SNAPBACK}][/a]


You also get artificial results, as people might listen again and again, paying much more attention than they normally would.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: eisa01 on 2005-12-07 00:53:05
My fear is that I'll only be able to figure out what the low anchor is

Have only tested two samples, and I've only managed to find one version that had artifacts.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: markanini on 2005-12-07 01:28:33
So far I've done three samples and sure it's difficult to spot artefacts on other than the low anchor but it's not totally impossible. A quiet enviroment and adjusting the volume moderately loud for each sample helps. The hard parts is probably that you need to be in a focused state of mind and you'll have to listen to all the samples more than a few times to spot the artefacts.

Just a few observations I made. Not that I've done many listening teste before...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Oliver on 2005-12-07 02:09:41
Quote
My fear is that I'll only be able to figure out what the low anchor is
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=348195")
Hi eisa01,

I have the same fear!  I plan to use the practice files, in the hope that they will improve my ability to hear the problems.

Oliver

Practice files [a href="http://ff123.net/64test/64kbsPractice.zip]http://ff123.net/64test/64kbsPractice.zip[/url]
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Dibrom on 2005-12-07 02:12:42
It has already been requested twice that people stop discussing their experience of the listening test itself while the still is still being carried out.

Please respect that, otherwise it is likely that this thread will be closed so as to maintain test integrity.

Really, until the test is finished, the only things that should be discussed in this thread, in my opinion (Sebastian can add something here if he doesn't agree), are metaissues such as discussion about obtaining or distributing the files, announcing the test at other places, or discussion about problems with the abc/hr program.

If you can't hear any differences, fine.  That is just as useful of a result for the purposes of the test.  Please don't ask other people to tell you if they hear differences or to describe their testing experience.

By the way, if Sebastian requests it, I will remove previous posts discussing any of this from the current thread.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-07 03:10:33
I found an issue with the delivered ABC/HR program :

DISCLAIMER - DO NOT READ THIS IF YOU ARE PERFORMING THE OPEN TEST - IT MAY INFLUENCE YOUR RESULTS.

PM me for details.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ErikS on 2005-12-07 03:32:40
Quote
I found an issue with the delivered ABC/HR program :
[...]hear a click[...]
[a href="index.php?act=findpost&pid=348218"][{POST_SNAPBACK}][/a]

Make sure you're using JRE v1.5. If you haven't upgraded, clicks in the sound is an issue which is already known. Perhaps there should have been a warning or something when abc/hr falls back on older JRE... well well...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-07 04:03:35
I am using the latest JRE. I reformatted two days ago.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 05:51:45
Quote
I am using the latest JRE. I reformatted two days ago.
[a href="index.php?act=findpost&pid=348229"][{POST_SNAPBACK}][/a]


Me too.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 05:54:31
Quote
Quote
If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.[a href="index.php?act=findpost&pid=348183"][{POST_SNAPBACK}][/a]


You also get artificial results, as people might listen again and again, paying much more attention than they normally would.
[a href="index.php?act=findpost&pid=348194"][{POST_SNAPBACK}][/a]


But then again, that is true for any listening tests. Or do you suggest we just listen to every sample once, as if we were only 'enjoying the music', and then rate everything at 5.0?

However, I will respect Sebastians request not to ask/answer questions like that. And nobody has gone against him since he stated his opinion on the matter.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 05:59:09
Quote
Quote
Quote
If this is not the case it would be an improvement for the test results IMHO -- because you get real ratings instead of all 5.0s.[a href="index.php?act=findpost&pid=348183"][{POST_SNAPBACK}][/a]


You also get artificial results, as people might listen again and again, paying much more attention than they normally would.
[a href="index.php?act=findpost&pid=348194"][{POST_SNAPBACK}][/a]


But then again, that is true for any listening tests. Or do you suggest we just listen to every sample once, as if we were only 'enjoying the music', and then rate everything at 5.0?

However, I will respect Sebastians request not to ask/answer questions like that. And nobody has gone against him since he stated his opinion on the matter.
[a href="index.php?act=findpost&pid=348240"][{POST_SNAPBACK}][/a]


No, it's OK to re-listen to the song as many times as you want.
However, when someone says that sample XYZ has an artifact, you are not going to let go until you spotted it because you know there is an artifact there.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Lyx on 2005-12-07 06:03:24
Quote
But then again, that is true for any listening tests. Or do you suggest we just listen to every sample once, as if we were only 'enjoying the music', and then rate everything at 5.0?

This is not a black/white issue. Some "artificial" training is necessary - but the line has to be drawn somewhere. And you now know where it is.

- Lyx
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: plonk420 on 2005-12-07 06:57:54
RareWares seems to be sending .TORRENT files as wrong MIME type. browser normally opens them to download or launch (and trust me, i use a LOT of torrent sites ;). RareWares causes my firefox to display them as text files...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: SpacemanSpiff0x0 on 2005-12-07 07:01:45
Holy crap, I just finished the test and I must say, I am really impressed by the quality of these samples. It's phenominal.

I have one question and I will not in any way be aggrevated if it is not answered (for some test reason).

The question I have is, what were the specific settings for the Nero encoded files. Was it one of the default LC profiles. Perhaps VBR/Stereo - Transparent [110-150]? Or was it a custom setting?

Thanks.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-07 07:09:38
Quote
I have one question and I will not in any way be aggrevated if it is not answered (for some test reason).
[a href="index.php?act=findpost&pid=348253"][{POST_SNAPBACK}][/a]

You can read the whole discussion about the choice of samples in the pre-test thread. just search for it.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: caligae on 2005-12-07 08:09:32
Quote
I found an issue with the delivered ABC/HR program :


I don't think you can avoid this (except every listener being supervised by some authority during the test). Even if you use strong encryption, the sound has to go to the soundcard unencrypted sooner or later. It's just like with DRM.

Yesterday I could easily abx a file without even the sound turned on within a few seconds.

So we just have to trust that no one wants to ruin the test by sending in manipulated results. Since the encoders and settings are also available, it would even be possible to favor/bash a specific codec.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-07 08:26:46
Quote
I don't think you can avoid this (except every listener being supervised by some authority during the test). Even if you use strong encryption, the sound has to go to the soundcard unencrypted sooner or later. It's just like with DRM.
[a href="index.php?act=findpost&pid=348261"][{POST_SNAPBACK}][/a]

It would be possible to implement a disk cache flushing mechanism that would make sure that both accesses take the same time.  Or, on opposite levels, a cache mechanism that would make sure that no delays are heard when playing back sound.  A sort of safety, like, don't start playing until all files are fully cached (at least the beginning of them, maybe 1-2 seconds, praps)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Tiis on 2005-12-07 08:29:12
Dosent't work for me..

Decompressed ABC-HR, than copyed and decompressed the Sample03 in the ABC-HR folder and run the batch for Sample03 (took me 5min searching around to find the batches, a hint in the readme.htm would be nice)

Then I loaded the sample03config and tried... no play button is working correctly, there is no sound coming out of my headphone or speakers!
Foobar ist running perfectly..

Im using WinXp Pro Sp2, Audigy2 ZS 2.08 Drivers and Java 1.5.6

Edit:
Switching the Playbackdevice to the second Audigy Entry did the job!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sTisTi on 2005-12-07 08:33:19
Quote
Then I loaded the sample03config and tried... no play button is working correctly, there is no sound coming out of my headphone or speakers!
Foobar ist running perfectly..
[a href="index.php?act=findpost&pid=348265"][{POST_SNAPBACK}][/a]

I had the same issue at the beginning. I changed the output device from "<Name Of My Soundcard>" to "primary device" (or the other way round, don't remember) in the ABC/HR options, then it worked.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: caligae on 2005-12-07 13:29:50
Quote
Quote
I don't think you can avoid this (except every listener being supervised by some authority during the test). Even if you use strong encryption, the sound has to go to the soundcard unencrypted sooner or later. It's just like with DRM.
[a href="index.php?act=findpost&pid=348261"][{POST_SNAPBACK}][/a]

It would be possible to implement a disk cache flushing mechanism that would make sure that both accesses take the same time.  Or, on opposite levels, a cache mechanism that would make sure that no delays are heard when playing back sound.  A sort of safety, like, don't start playing until all files are fully cached (at least the beginning of them, maybe 1-2 seconds, praps)
[a href="index.php?act=findpost&pid=348264"][{POST_SNAPBACK}][/a]

Of course that specific "click" issue can be fixed more or less easily. What I meant is that there will always be a way of successfully ABXing a track without actually hearing a difference.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Raptus on 2005-12-07 13:32:13
I'm having some issues with ABC/HR, too.

- after opening several sessions I get short drop outs in sound (garbage collector issue?), forcing me to restart ABC/HR
- after some inactive time (like when you have to eat between tests  ) moving the sliders won't update the rank number anymore
- when I start ABC/HR I get a blank window in most cases. With some luck I can open it after restarting the PC...

Latest JRE 1.5.0 under W2K.

Apart from that I'm two thirds through the test 
I'm unhappy with some sample choices but I'll leave that for later.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: guruboolez on 2005-12-07 17:23:50
Quote
In the last Guru's test  difference also was about 10% between result of Nero and Itunes.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=348131")

 

CLASSICAL
AAC iTunes  133,33 kbps
AAC Nero    125,71 kbps

NON-CLASSICAL (short samples)
AAC iTunes  137,31 kbps
AAC Nero   134,10 kbps

[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38792]http://www.hydrogenaudio.org/forums/index....showtopic=38792[/url]

The difference is ~6% with classical and only 2,40% for non-classical (on average: I still forgot to add the bitrate table).
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Ivan Dimkovic on 2005-12-07 17:42:17
@Guru - unfortunately bit-rate measurement from your previous test is not very relevant for the Nero encoder used in this test, as it is completely different.

Also, @All - I did Advanced-PEAQ analysis (with Opticom Opera) of the listening test samples, and did statistical analysis afterwards (ANOVA).

I will publish results when the test is over - it will be very interesting to measure correlation between Advanced PEAQ and real-world listening test results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: guruboolez on 2005-12-07 17:52:29
Quote
@Guru - unfortunately bit-rate measurement from your previous test is not very relevant for the Nero encoder used in this test, as it is completely different.[a href="index.php?act=findpost&pid=348354"][{POST_SNAPBACK}][/a]

I know. I just answered to IgorC which didn't give correct information about my last personal listening test.

Furthemore, I don't agree with this: “It's not hard to assume iTunes will be better on this test due to quite big oversize”. During latest listening test, Nero Digital encodings were oversized compared to iTunes CBR, and the latter performed better. The assumption: "higher bitrate -> higher quality" isn't necessary true.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: SpacemanSpiff0x0 on 2005-12-07 19:25:01
Ok, so i went to the other thread and searched for the settings.

Quote
# Settings are:

iTunes 6.0.1.3: 128 kbps, VBR
LAME 3.97b2: -V5 --vbr-new --noreplaygain
Nero 3.1.0.2 (only I have it ATM): Streaming Profile (make sure LC is selected!)
Shine 0.1.4: -b 128
AoTuV 4.51: -q 4.25 (or 4,25 depending on system settings)
WMA Pro 9.1 (using VBS): -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24


One thing I noticed is that when Sebastian talked about the Nero files he mentioned that it was ABR. I was under the impression that it was VBR only.

Another question I had was why not use the built in "Calculate" function to produce a file that was truly 128Kb average?

If these are ignorant questions, I apologize, I'm only recently involved in these types of things.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 19:51:11
Quote
took me 5min searching around to find the batches, a hint in the readme.htm would be nice
[a href="index.php?act=findpost&pid=348265"][{POST_SNAPBACK}][/a]


It was mentioned in 4. all the time.

Quote
Ok, so i went to the other thread and searched for the settings.

Quote
# Settings are:

iTunes 6.0.1.3: 128 kbps, VBR
LAME 3.97b2: -V5 --vbr-new --noreplaygain
Nero 3.1.0.2 (only I have it ATM): Streaming Profile (make sure LC is selected!)
Shine 0.1.4: -b 128
AoTuV 4.51: -q 4.25 (or 4,25 depending on system settings)
WMA Pro 9.1 (using VBS): -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24


One thing I noticed is that when Sebastian talked about the Nero files he mentioned that it was ABR. I was under the impression that it was VBR only.

Another question I had was why not use the built in "Calculate" function to produce a file that was truly 128Kb average?

If these are ignorant questions, I apologize, I'm only recently involved in these types of things.
[a href="index.php?act=findpost&pid=348376"][{POST_SNAPBACK}][/a]


The "Streaming" profile was used because it was recommended by the developers. Also, Juha mentioned something that it uses an updated core while the other settings don't.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: SpacemanSpiff0x0 on 2005-12-07 19:57:34
Oh, thanks for the info!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 20:01:33
I am receiving some e-mails from users who are bitching that the test is useless and that calling the test "... @ 128 kbps" is a lie because we are not even testing at 128 kbps.

I would like to make it clear again that the 128 kbps are supposed to be reached on a large collection of audio tracks and not on 18, usually hard-to-encode, 30 seconds samples. It's the nature of VBR to encode difficult parts of a song at a higher bitrate than less complex parts. Therefore, even when encoding one single track, chances are good to reach something like 128 kbps because the full track is usually a mixture of hard to encode parts and easy to encode parts. Our samples however contain the difficult parts of an audio track only.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 20:02:50
Quote
Oh, thanks for the info!
[a href="index.php?act=findpost&pid=348385"][{POST_SNAPBACK}][/a]


One more thing - according to Ivan, the encoder produces bitrates around 134 kbps which I can confirm after letting it encode my small music collection of 300 files. And 134 kbps is pretty close to 128 kbps.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 20:34:17
Congrats to Raptus for finishing the whole sample set. Thanks!

Also, thanks to the rest who also submitted results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Raptus on 2005-12-07 21:10:01
Quote
Congrats to Raptus for finishing the whole sample set. Thanks!

was I the first one to finish them all?
6 hours straight with pauses, now im exhausted 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 21:12:26
Quote
Quote
Congrats to Raptus for finishing the whole sample set. Thanks!

was I the first one to finish them all?
6 hours straight with pauses, now im exhausted 
[a href="index.php?act=findpost&pid=348407"][{POST_SNAPBACK}][/a]


Yep.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-07 21:27:13
Quote
I am receiving some e-mails from users who are bitching that the test is useless and that calling the test "... @ 128 kbps" is a lie because we are not even testing at 128 kbps.

I would like to make it clear again that the 128 kbps are supposed to be reached on a large collection of audio tracks and not on 18, usually hard-to-encode, 30 seconds samples. It's the nature of VBR to encode difficult parts of a song at a higher bitrate than less complex parts. Therefore, even when encoding one single track, chances are good to reach something like 128 kbps because the full track is usually a mixture of hard to encode parts and easy to encode parts. Our samples however contain the difficult parts of an audio track only.
[a href="index.php?act=findpost&pid=348387"][{POST_SNAPBACK}][/a]



OMFG! Julius!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 21:28:26
Quote
Quote
I am receiving some e-mails from users who are bitching that the test is useless and that calling the test "... @ 128 kbps" is a lie because we are not even testing at 128 kbps.

I would like to make it clear again that the 128 kbps are supposed to be reached on a large collection of audio tracks and not on 18, usually hard-to-encode, 30 seconds samples. It's the nature of VBR to encode difficult parts of a song at a higher bitrate than less complex parts. Therefore, even when encoding one single track, chances are good to reach something like 128 kbps because the full track is usually a mixture of hard to encode parts and easy to encode parts. Our samples however contain the difficult parts of an audio track only.
[a href="index.php?act=findpost&pid=348387"][{POST_SNAPBACK}][/a]



OMFG! Julius!
[a href="index.php?act=findpost&pid=348416"][{POST_SNAPBACK}][/a]


Exactement. 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 21:35:20
Quote
Quote

OMFG! Julius!
[a href="index.php?act=findpost&pid=348416"][{POST_SNAPBACK}][/a]


Exactement. 
[a href="index.php?act=findpost&pid=348418"][{POST_SNAPBACK}][/a]


Sounds like HA.org has its own house troll...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-07 21:44:28
Does anyone know how to post on Shlashdot? I am pretty lost.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 22:17:10
Quote
Does anyone know how to post on Shlashdot? I am pretty lost.
[a href="index.php?act=findpost&pid=348428"][{POST_SNAPBACK}][/a]


Do we really want that?

Only kidding. I could do it.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-07 22:17:47
Quote
Quote
Quote

OMFG! Julius!
[a href="index.php?act=findpost&pid=348416"][{POST_SNAPBACK}][/a]


Exactement. 
[a href="index.php?act=findpost&pid=348418"][{POST_SNAPBACK}][/a]


Sounds like HA.org has its own house troll...
[a href="index.php?act=findpost&pid=348424"][{POST_SNAPBACK}][/a]


Nah, he's more like a homeless troll from the r3mix days.

He posted here only once, and was banned on sight. Claims to have a proxy'ed account now...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 22:42:38
Quote
Quote
Does anyone know how to post on Shlashdot? I am pretty lost.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=348428")


Only kidding. I could do it.
[a href="index.php?act=findpost&pid=348435"][{POST_SNAPBACK}][/a]


Ok. Here's a suggestion. Unless anyone has any strong objections or suggestions I will post this in 20 minutes:

(Converted to forum code.)

[a href="http://maresweb.org/]Sebastian Mares[/url] has launced a new audio encoder test (http://www.hydrogenaudio.org/forums/index.php?showtopic=39448) for the 128 kbps range. The listening test includes Ogg Vorbis, MP3, AAC (Nero and iTunes) and WMA Pro - all using the latest and greatest encoders available.

Some self-proclaimed audio 1337 may think that 128 kbps isn't worth the effort, but having given the test a try myself I can say that distinguishing 128 kbps encodes from the originals is no simple task  these days. Dig out those golden ears and go help them out (http://www.maresweb.de/listening-tests/mf_128_1.php) - the more testers the merrier.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-07 22:52:40
Sebastian Mares has launcHed

Other than that, sounds excellent
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-07 22:57:10
I think you should also say that they're VBR ranges, and they remain RANGES, not exact bitrates - they spread out on collections, etc.. what has been said on this topic, to prevent seb's email getting spammed with "USELESS, not 128!!"
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 23:17:12
Quote
I think you should also say that they're VBR ranges, and they remain RANGES, not exact bitrates - they spread out on collections, etc.. what has been said on this topic, to prevent seb's email getting spammed with "USELESS, not 128!!"
[a href="index.php?act=findpost&pid=348450"][{POST_SNAPBACK}][/a]


I could try adding a little something about that. Not to much though - people wont ever get through it all...

@Rjamorim: noted.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-07 23:27:08
Quote
Quote
,Dec 8 2005, 12:57 AM]I think you should also say that they're VBR ranges, and they remain RANGES, not exact bitrates - they spread out on collections, etc.. what has been said on this topic, to prevent seb's email getting spammed with "USELESS, not 128!!"
[a href="index.php?act=findpost&pid=348450"][{POST_SNAPBACK}][/a]


I could try adding a little something about that. Not to much though - people wont ever get through it all...

@Rjamorim: noted.
[a href="index.php?act=findpost&pid=348459"][{POST_SNAPBACK}][/a]


Ok. Submitted now with a few slight modifications, as well as a line about the VBR aspect.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2005-12-08 00:25:04
Guru  It won´t be a problem if I´d call you Guru. It´s hard to not say incorrectly your nickname. 
I didn´t interprete myself correct. I mean there is 10%  difference between  your  objective  marks. For example 4 and 4.4. And not between bitrates.
I wanted to say that 10% is not a tiny difference for bitrate and  objective marks. Btw the difference reach up to 17% and not 10%.  . It maybe sound  too precise and too annoying. But its not 1-2 % like in Guru test  ( Guru´s test is enough precise for real life ). It´s up to 17%.


But this test wont be enough precise for people like from Doom9 (video entuasist) who search a good  audio codec for ABR encodes.   

However  it test  will be usefull.
I  think if new Nero and itunes will be tight I will choose Nero cause of robust file size.  I´m understanding that people here not care much about final size.

The idea about VBR doesn´t  convince me at all. I didn´t see any scientific or probalistic (or both) proof  about that if one codec always produces a big file size (as it happens in this test) on VBR then it will most smart on ABR.     
Ok, for melomans ( audiofils, sound professionals etc ) the size isn´t problem,  but there a lot of people who care about it.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: jido on 2005-12-08 00:37:02
I uploaded the MacOS X (Tiger) version of the bin folder. Enjoy!

http://www.hydrogenaudio.org/forums/index....showtopic=39523 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39523)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Pio2001 on 2005-12-08 01:21:13
Quote
if one codec always produces a big file size (as it happens in this test) on VBR then it will most smart on ABR.    [a href="index.php?act=findpost&pid=348474"][{POST_SNAPBACK}][/a]


They say that codecs do not produce always a big file size, except in this test.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-08 02:13:32
Quote
Really, until the test is finished, the only things that should be discussed in this thread, in my opinion (Sebastian can add something here if he doesn't agree), are metaissues such as discussion about obtaining or distributing the files, announcing the test at other places, or discussion about problems with the abc/hr program.[a href="index.php?act=findpost&pid=348204"][{POST_SNAPBACK}][/a]

I agree on this. If someone has technical problems the possibly available answers would be easier to find without the OT clutter.

We already had a thread of 865 posts about the codec choices and other details. This thread should really concentrate on the topic. People who are invited to take the test and come here may get confused.

I would like to see all OT replies moved to another thread (including this one).

[span style='font-size:7pt;line-height:100%']Edit: added the quote (Argh, now I am doing it myself, adding clutter...)[/span]
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-08 02:37:44
Quote
We already had a thread of 865 posts about the codec choices and other details. This thread should really concentrate on the topic. People who are invited to take the test and come here may get confused.
[a href="index.php?act=findpost&pid=348496"][{POST_SNAPBACK}][/a]

In any case, the only information you need about the test is in the first post.  The rest is for follow-up only.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-08 10:23:38
My slashdot posting was rejected...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Shade[ST] on 2005-12-08 10:27:35
Quote
My slashdot posting was rejected...
[a href="index.php?act=findpost&pid=348578"][{POST_SNAPBACK}][/a]

Duplicates, maybe?  Someone mentioned that posting on Slashdot was planned?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-08 11:00:30
Quote
Quote
My slashdot posting was rejected...
[a href="index.php?act=findpost&pid=348578"][{POST_SNAPBACK}][/a]

Duplicates, maybe?  Someone mentioned that posting on Slashdot was planned?
[a href="index.php?act=findpost&pid=348579"][{POST_SNAPBACK}][/a]


Don't know. Maybe they just didn't find it interesting enough. (I posted to index, as 'announcement'.)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-08 12:21:18
Quote
My slashdot posting was rejected...[a href="index.php?act=findpost&pid=348578"][{POST_SNAPBACK}][/a]


Makes me wonder if that wasn't for the best...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-08 12:24:13
Quote
Quote
My slashdot posting was rejected...[a href="index.php?act=findpost&pid=348578"][{POST_SNAPBACK}][/a]


Makes me wonder if that wasn't for the best...
[a href="index.php?act=findpost&pid=348599"][{POST_SNAPBACK}][/a]


Heh. You're probably right 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: minisu on 2005-12-08 20:32:48
I think that there are too many samples, at least for me. I don't got the time/patience to test all samples.

Anyway, I sent the results for the first two samples. I could point out 3 of 6 codecs. It'll be very interesting to know which they are. I'm especially interesting in my results on WMA Pro since I haven't tested it before.

Have there been a major listening test with a recent WMA Pro?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-08 20:51:03
Not that I know of. Roberto used WMA Pro once, but it was 9.0 if I am not mistaken.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: rjamorim on 2005-12-08 23:17:14
Quote
Not that I know of. Roberto used WMA Pro once, but it was 9.0 if I am not mistaken.[a href="index.php?act=findpost&pid=348704"][{POST_SNAPBACK}][/a]


Yes. And it was the oh-so-dreaded two-pass VBR profile. Makes me wonder why noone bitched at me. Must be because I'm much less polite than you and I have no patience for that sort of bullshit :B
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: babaz on 2005-12-08 23:24:44
hallo everybody
please take my request as easy as possible
since i hate java, expecially having to install a huge JRE just to use this program (annd maybe a very few others), is it possible to have a win32 compiled binary as a standalone .exe?

would be great 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Pio2001 on 2005-12-09 02:08:50
I installed the JRE in order to be in the conditions required for the test, but the jar file seemed to work without the JRE. I have Windows XP Pro SP2.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: saratoga on 2005-12-09 03:37:35
Quote
I installed the JRE in order to be in the conditions required for the test, but the jar file seemed to work without the JRE. I have Windows XP Pro SP2.
[a href="index.php?act=findpost&pid=348784"][{POST_SNAPBACK}][/a]


You need a JRE (or the JDK) to run Java.  It doesn't have to be the Sun one though (in theory, lots of stuff requires the Sun one).
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Althalus on 2005-12-09 15:32:33
Unable to run test on OSX

Get error when opening the .ecf

Config file could not be loaded.
File Sample01/BigYellow.wav could not be opened.

Should mention that after running the bin_osx/Sample01.command it extracted the .zip to a Sample01 folder, but files therein have different names than the one in the error message:

/Sample01
BigYellow_1.wav
BigYellow_2.wav
BigYellow_3.wav
BigYellow_4.wav
BigYellow_5.wav

Any tips?

Note: tried to listen to the this sample using my e3c's and must say that my ears apparently suck a good deal more than i expected
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-09 16:12:12
BigYellow.wav is the reference file. I have no idea how to fix the problem since I have 0 experience with OS X. Maybe jido, who created/modified the bash scripts can help you and send me an updated version of the .command files if he sees need.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Pio2001 on 2005-12-09 17:52:51
You're missing two files in your directory : BigYellow.wav and BigYellow_6.wav
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-09 18:01:18
Looks like a problem decoding WavPack then...

I am going to remove the scripts for OS X until a fix is found.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: OverlordQ on 2005-12-10 00:40:59
Quote
My slashdot posting was rejected...
[a href="index.php?act=findpost&pid=348578"][{POST_SNAPBACK}][/a]


Once the results come in, post that, you'd most definately not want the /. masses praticipating since 9/10 they'd just purposely screw up the stats.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Halcyon on 2005-12-10 10:20:26
Quote
Once the results come in, post that, you'd most definately not want the /. masses praticipating since 9/10 they'd just purposely screw up the stats.


Interesting. How would they purposely screw up the stats?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: jido on 2005-12-10 10:37:41
Quote
Unable to run test on OSX

Get error when opening the .ecf

Config file could not be loaded.
File Sample01/BigYellow.wav could not be opened.

Should mention that after running the bin_osx/Sample01.command it extracted the .zip to a Sample01 folder, but files therein have different names than the one in the error message:

/Sample01
BigYellow_1.wav
BigYellow_2.wav
BigYellow_3.wav
BigYellow_4.wav
BigYellow_5.wav

Any tips?

Note: tried to listen to the this sample using my e3c's and must say that my ears apparently suck a good deal more than i expected
[a href="index.php?act=findpost&pid=348949"][{POST_SNAPBACK}][/a]

Do you have MacOS X 10.4 Tiger?
You should have a "lib" folder in bin_osx, otherwise wavpack will not decode files.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-10 23:03:32
Quote
Quote
Once the results come in, post that, you'd most definately not want the /. masses praticipating since 9/10 they'd just purposely screw up the stats.


Interesting. How would they purposely screw up the stats?
[a href="index.php?act=findpost&pid=349130"][{POST_SNAPBACK}][/a]


Maybe he thinks they would just rate the samples at random?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-11 07:41:44
Quote
Quote
Quote
Once the results come in, post that, you'd most definately not want the /. masses praticipating since 9/10 they'd just purposely screw up the stats.


Interesting. How would they purposely screw up the stats?
[a href="index.php?act=findpost&pid=349130"][{POST_SNAPBACK}][/a]


Maybe he thinks they would just rate the samples at random?
[a href="index.php?act=findpost&pid=349263"][{POST_SNAPBACK}][/a]


Chances are good that they would rate the reference sample then, too. Such results are easy to sort out.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-11 08:25:43
Quote
hey guys.

just finished setting up all the files needed for the test including installation of java, and i have a problem. the wav's dont play. what am i doing wrong?
reference file doesnt play, ABX option playback doesnt work either. checked settings and my sound card is selected.....
need help if you want some results

thanks all
[a href="index.php?act=findpost&pid=348010"][{POST_SNAPBACK}][/a]


What sound card is selected?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Althalus on 2005-12-11 11:40:56
Quote
Do you have MacOS X 10.4 Tiger?
You should have a "lib" folder in bin_osx, otherwise wavpack will not decode files.
[a href="index.php?act=findpost&pid=349133"][{POST_SNAPBACK}][/a]

Yes 10.4.3 - Can't remember if there was a lib folder. (Tried to run test at work, now weekend )
If it's not there, should I download it seperately somewhere or just create an empty lib folder?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-11 12:04:45
Quote
Quote

Do you have MacOS X 10.4 Tiger?
You should have a "lib" folder in bin_osx, otherwise wavpack will not decode files.
[a href="index.php?act=findpost&pid=349133"][{POST_SNAPBACK}][/a]

Yes 10.4.3 - Can't remember if there was a lib folder. (Tried to run test at work, now weekend )
If it's not there, should I download it seperately somewhere or just create an empty lib folder?
[a href="index.php?act=findpost&pid=349390"][{POST_SNAPBACK}][/a]


A "lib" folder was included inside "bin_osx". It contained the following files:

libwavpack.0.0.0.dylib
libwavpack.0.dylib
libwavpack.a
libwavpack.dylib
libwavpack.la
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: singaiya on 2005-12-11 14:07:06
I've downloaded the torrent with all samples but can't find the "bin" directory referenced in the readme. I already had ABC/HR Java from before so I didn't download that again. Also there's no .bat files, just the audio samples. Is the stuff I'm missing in the ABC package you're hosting?

edit: Nevermind, I DL'd your ABC zip and everything is in it. You may want to clarifiy that on the readme. As it is now, it looks like you don't need to download it if you already have it.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Serge Smirnoff on 2005-12-11 23:54:50
Quote
Also, @All - I did Advanced-PEAQ analysis (with Opticom Opera) of the listening test samples, and did statistical analysis afterwards (ANOVA).

I will publish results when the test is over - it will be very interesting to measure correlation between Advanced PEAQ and real-world listening test results.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=348354")


I’ve also finished loading the same contenders but different sound samples to SoundExpert testing engine. Starting ratings based on my own listening tests are ready but will be published at the same time with the current test results. Corresponding web page [a href="http://www.soundexpert.info/coders128.jsp]“Special 128 kbit/ test”[/url] is ready now.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Althalus on 2005-12-12 07:21:02
Quote
Quote
Quote
Do you have MacOS X 10.4 Tiger?
You should have a "lib" folder in bin_osx, otherwise wavpack will not decode files.
[a href=\"index.php?act=findpost&pid=349133\"][{POST_SNAPBACK}][/a]
Yes 10.4.3 - Can't remember if there was a lib folder. (Tried to run test at work, now weekend )
If it's not there, should I download it seperately somewhere or just create an empty lib folder?
[a href=\"index.php?act=findpost&pid=349390\"][{POST_SNAPBACK}][/a]

A "lib" folder was included inside "bin_osx". It contained the following files:

libwavpack.0.0.0.dylib
libwavpack.0.dylib
libwavpack.a
libwavpack.dylib
libwavpack.la
[a href=\"index.php?act=findpost&pid=349393\"][{POST_SNAPBACK}][/a]

OK, issue solved, following is dump of output when running the .command.

Code: [Select]
Last login: Mon Dec 12 07:13:00 on ttyp1
Welcome to Darwin!
dhcp1s608:~ az$ /Users/az/Desktop/ABC-HR/bin_osx/Sample01.command; exit
/Users/az/Desktop/ABC-HR/Sample01.zip
Archive:  Sample01.zip
   creating: Sample01/
  inflating: Sample01/BigYellow.wv  
  inflating: Sample01/BigYellow_1.m4a  
  inflating: Sample01/BigYellow_2.mp3  
  inflating: Sample01/BigYellow_3.mp4  
  inflating: Sample01/BigYellow_4.mp3  
  inflating: Sample01/BigYellow_5.ogg  
  inflating: Sample01/BigYellow_6.wv  
dyld: Library not loaded: /usr/local/lib/libwavpack.0.dylib
  Referenced from: /Users/az/Desktop/ABC-HR/bin_osx/wvunpack
  Reason: image not found
/Users/az/Desktop/ABC-HR/bin_osx/Sample: line 38: 10196 Trace/BPT trap          $wvunpack ${basename}.wv
 ****** FAAD2 (Freeware AAC Decoder) V2.0 RC1  ******

        Build: Sep 11 2003
        Copyright: M. Bakker
                   Ahead Software AG
        http://www.audiocoding.com
        Floating point version

 ****************************************************

BigYellow_1.m4a file info:
Track   Type    Info
1       audio   MPEG-4, 24.543 secs, 128 kbps, 44100 Hz

  ---------------------
 | Config:  2 Ch       |
  ---------------------
 | Ch |    Position    |
  ---------------------
 | 00 | Left front     |
 | 01 | Right front    |
  ---------------------

Decoding BigYellow_1.m4a took:  0.58 sec.
input:  BigYellow_2.mp3  (44.1 kHz, 2 channels, MPEG-1 Layer III)
output: BigYellow_2.wav  (16 bit, Microsoft WAVE)
skipping initial 1105 samples (encoder+decoder delay)
Frame#   940/940     32 kbps   MS            
 ****** FAAD2 (Freeware AAC Decoder) V2.0 RC1  ******

        Build: Sep 11 2003
        Copyright: M. Bakker
                   Ahead Software AG
        http://www.audiocoding.com
        Floating point version

 ****************************************************

BigYellow_3.mp4 file info:
Track   Type    Info
1       audio   MPEG-4, 24.517 secs, 136 kbps, 44100 Hz
2       od      Object Descriptors
3       scene   BIFS

  ---------------------
 | Config:  2 Ch       |
  ---------------------
 | Ch |    Position    |
  ---------------------
 | 00 | Left front     |
 | 01 | Right front    |
  ---------------------

Decoding BigYellow_3.mp4 took:  0.58 sec.
input:  BigYellow_4.mp3  (44.1 kHz, 2 channels, MPEG-1 Layer III)
output: BigYellow_4.wav  (16 bit, Microsoft WAVE)
skipping initial 1105 samples (encoder+decoder delay)
Frame#   930/930    128 kbps                
OggDec 1.0.1
Decoding "BigYellow_5.ogg" to "BigYellow_5.wav"
        [100.0%]
dyld: Library not loaded: /usr/local/lib/libwavpack.0.dylib
  Referenced from: /Users/az/Desktop/ABC-HR/bin_osx/wvunpack
  Reason: image not found
/Users/az/Desktop/ABC-HR/bin_osx/Sample: line 38: 10203 Trace/BPT trap          $wvunpack ${basename}_6.wv
logout
[Process completed]
By copying the content of bin_osx/lib to /usr/local/lib it decodes perfectly fine.
Someone might either fix the script or add fixnotes to the readme under a OSX section.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-12 12:17:39
OK, I will edit the readme files once I get back home.

PS: Please use codebox tags instead of code when posting much data.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: eisa01 on 2005-12-14 19:17:55
I've done all the samples now, but I'm not so sure if I should submit my results. The thing is, I'm quite sure I've only managed to identify the low anchor in the tests, so all the other sample have got 5 from me. And I think I rated the low sample a little too high in the start. Do you want my results?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-14 19:30:10
All results are welcome.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kdo on 2005-12-17 21:16:23
I've got a problem with playback in the java/abchr.

My setup: Win2000. Installed latest 1.5 Java today.
I have some ESS Maestro on a laptop. Pretty crappy sound setup, I know.

The problem occurs when I select "Primary sound dirver" or "ESS Maestro" in the abchr settings.

1) if I simply click to play something it plays ok, but the sound is muffled.

I get the same effect if I select the "Direct sound" and "Allow hardware mixing" in Foobar. (Foobar says: "Device info: hardware mixing available" but apparently this option doesn't work on my setup). In Foobar I'm using "Direct sound" without hardware mixing, and it sounds fine. Foobar "WaveOut" works too but it skips sometimes.


2) so java/abchr plays muffled, but then if I click to switch A or B or Ref - it also changes the sound - from muffled to what I think is normal sound. Then if I switch (EDIT: I mean I switch during playback) A,B,Ref again it switches back to muffled and so on:
it always starts with muffled, then normal,muffled,normal,muffled,.....


3) There are two more choices available in the abchr settings:
the default "Port ESS Maestro" doesn't produce any sound.
"Java sound engine" sounds normal but much quieter and it skips when switching A-B.


Note: this is not very urgent problem. However I am planning to try the next 48 kbps test. (Edit2: deleted unnecessary remark about my abx-ability in this test)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: plonk420 on 2005-12-18 04:44:19
Quote
Quote
hey guys.

just finished setting up all the files needed for the test including installation of java, and i have a problem. the wav's dont play. what am i doing wrong?
reference file doesnt play, ABX option playback doesnt work either. checked settings and my sound card is selected.....
need help if you want some results

thanks all
[a href="index.php?act=findpost&pid=348010"][{POST_SNAPBACK}][/a]


What sound card is selected?
[a href="index.php?act=findpost&pid=349356"][{POST_SNAPBACK}][/a]


my ABCHR selected Bluetooth Audio by default (and my Skype and Gizmo like to choose the soundcard as i/o device  as of late)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: [JAZ] on 2005-12-18 12:22:15
Mmmm.... This remembers me of an old problem with Soundblasters, with directsound.

If it used primary buffers, it played at 22.5khz. I doubt this is the same problem, but it would be a good start to investigate.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: OverlordQ on 2005-12-20 08:04:09
Quote
Quote
Quote
Once the results come in, post that, you'd most definately not want the /. masses praticipating since 9/10 they'd just purposely screw up the stats.


Interesting. How would they purposely screw up the stats?
[a href="index.php?act=findpost&pid=349130"][{POST_SNAPBACK}][/a]


Maybe he thinks they would just rate the samples at random?
[a href="index.php?act=findpost&pid=349263"][{POST_SNAPBACK}][/a]


Hmmm what's that quote, never underestimate the power of idiots in large numbers?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-20 08:28:10
Quote
Quote
Quote
Quote
Once the results come in, post that, you'd most definately not want the /. masses praticipating since 9/10 they'd just purposely screw up the stats.


Interesting. How would they purposely screw up the stats?
[a href="index.php?act=findpost&pid=349130"][{POST_SNAPBACK}][/a]


Maybe he thinks they would just rate the samples at random?
[a href="index.php?act=findpost&pid=349263"][{POST_SNAPBACK}][/a]


Hmmm what's that quote, never underestimate the power of idiots in large numbers?
[a href="index.php?act=findpost&pid=351397"][{POST_SNAPBACK}][/a]


Yeah, but as I said, chances are good that they are going to rate the reference sample then, too.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-21 23:32:06
We are slowly approaching the deadline and I still don't have enough results for several samples.
Considering 10 results per sample are minimum, this is what I ask people to focus on :

Moved to first post: [a href="index.php?act=findpost&pid=347839"][{POST_SNAPBACK}][/a].
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Halcyon on 2005-12-22 09:58:48
Sorry, can't contribute at all with the Christmas coming and due to personal commitments elsewhere. I hope others can do better. I'll have to do the test for myself after the Christmas holidays. The results won't be useful for the test at large, but for personal learning it can be interesting.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: PoisonDan on 2005-12-22 10:11:08
Wel, Gambit already warned you about this earlier, didn't he? Most people's holidays (including mine) start with Christmas, so most people (including me) won't have time to perform this test until after Christmas.

Sorry.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-22 11:03:34
Well, I've done all samples 1-11. I'll send them in on the 25th if you don't extend the time. If you do I'll try to do the rest as well.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sTisTi on 2005-12-22 11:57:46
Quote
We are slowly approaching the deadline and I still don't have enough results for several samples.
[a href="index.php?act=findpost&pid=351772"][{POST_SNAPBACK}][/a]

After spending hours listening to the samples, I have to declare my defeat: besides some encodings of sample nr. 3, all other encodings are transparent to me (except for the low anchor of course, even I can distinguish that). No matter how intensely I listen, I seem to get fooled by all modern codecs at 128k - and I can't say I'm unhappy about that at all, to be honest 
However, to make more than one sample contribution, I've recruited my wife to take part in the test. She's the opposite of me WRT listening experience: She's a trained musician who plays several instruments but has no lossy artefacts training at all  She also started with sample 3 and was more successful than me right from the start. I'll try to convince her in the name of public good to try some more samples 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2005-12-22 12:12:32
Quote
We are slowly approaching the deadline and I still don't have enough results for several samples.
Considering 10 results per sample are minimum, this is what I ask people to focus on

i would test all of them. but i think you should extend it till 5 of january.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-22 18:41:08
Quote
Well, I've done all samples 1-11. I'll send them in on the 25th if you don't extend the time. If you do I'll try to do the rest as well.
[a href="index.php?act=findpost&pid=351858"][{POST_SNAPBACK}][/a]


It would be cool to have the results ASAP if you don't plan on testing additional samples. That saves me from sorting and evaluating a bunch of results at once - it's easier for me to sort four or five result sets per day.

The problem with extending the test is that I start school on January 6th and we're writing a couple of exams in the first two weeks, so I don't know how much time I am going to have. I will think about it until tomorrow evening.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Halcyon on 2005-12-22 20:07:27
I'm not sure how much a week's extension will help overall, but if your own schedules allow it, please consider it. I can promise to test half of the samples if there's an extension.

One thing to consider.

If you have to sit on the results without analysing for a couple of weeks, because of your own schedules, I think people will understand.

It's probably better to have more data (and higher reliability) with delayed reporting than on-schedule reporting, but lacking in data (and reliability). What do you think?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-23 07:05:49
Quote
I'm not sure how much a week's extension will help overall, but if your own schedules allow it, please consider it. I can promise to test half of the samples if there's an extension.

One thing to consider.

If you have to sit on the results without analysing for a couple of weeks, because of your own schedules, I think people will understand.

It's probably better to have more data (and higher reliability) with delayed reporting than on-schedule reporting, but lacking in data (and reliability). What do you think?
[a href="index.php?act=findpost&pid=351949"][{POST_SNAPBACK}][/a]


I am not sure if the lack of results is caused by people having to work or prepare themselves for Christmas / New Year, or simply because the test is hard at this bitrate. If the former applies, extending the test is a good idea. However, if the later applies, I see no point in extending the test just to obtain two or three additional results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-23 07:10:37
OK, let's be more precise...

Who could submit results if the test is extended by one or two weeks? So far, I have Halcyon, kwanbis and sTisTi (or his wife ).

BTW, if you don't plan testing all samples, it would be cool to test the samples for which I have too few results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: LANjackal on 2005-12-23 07:24:20
I could. I've been on vacation for a while now and have been unable to listen to the samples with my best equipment. The final decision is up to you, as always, since you're the one running the test .
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-23 11:36:09
Only 8 samples without enough results left.

I would also like to thank everyone who submitted until now.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Alex B on 2005-12-23 12:17:01
Quote
OK, let's be more precise...

Who could submit results if the test is extended by one or two weeks? So far, I have Halcyon, kwanbis and sTisTi (or his wife ).

BTW, if you don't plan testing all samples, it would be cool to test the samples for which I have too few results. [a href="index.php?act=findpost&pid=352019"][{POST_SNAPBACK}][/a]

Unfortunately I too have been quite busy with other activities. This test really needs concentration and a proper mood. I am going to post some results on 25th anyway, but an extension would make possible to add more samples.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2005-12-23 13:36:52
Quote
OK, let's be more precise...

Who could submit results if the test is extended by one or two weeks? So far, I have Halcyon, kwanbis and sTisTi (or his wife ).

BTW, if you don't plan testing all samples, it would be cool to test the samples for which I have too few results.
[a href="index.php?act=findpost&pid=352019"][{POST_SNAPBACK}][/a]

i could. and would try yo test them all.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: MedO on 2005-12-23 14:13:04
I'd try to test a few more samples if the test was extended (only came across it today).
However, on the first one my untrained ears were unable to rate anything but the low anchor, so I don't know how much of a contribution that'd be.

MedO
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: ErikS on 2005-12-23 16:18:35
Quote
OK, let's be more precise...

Who could submit results if the test is extended by one or two weeks? So far, I have Halcyon, kwanbis and sTisTi (or his wife ).

BTW, if you don't plan testing all samples, it would be cool to test the samples for which I have too few results.
[a href="index.php?act=findpost&pid=352019"][{POST_SNAPBACK}][/a]

Add me to that list. The testing was more demanding and took more time that I had thought, so I have unfortunately only done two samples so far. I promise to do at least two per day after the 26:th if you extend the deadline.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Caroliano on 2005-12-23 18:29:23
I also will try more samples if it get extended. Otherwise I will only make the last ones.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-23 22:00:17
Another problem when extending is that the time span between the end of my test and the beginning of the 48 kbps listening will become shorter - unless the person who is running the test doesn't mind postponing the start to February so there is a break of one month between the two.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sehested on 2005-12-23 22:45:03
Quote
Another problem when extending is that the time span between the end of my test and the beginning of the 48 kbps listening will become shorter - unless the person who is running the test doesn't mind postponing the start to February so there is a break of one month between the two.
[a href="index.php?act=findpost&pid=352109"][{POST_SNAPBACK}][/a]

I'm in support for an extension.

Waiting another week seems like a small sacrifice when compared to all the time that went into the preparation of this test.

Sebastian it seems that extending the test will provide significantly more results.

I would hate to see the conclusions of this test considered unreliable due to lack of test results. Just think of all the effort that went into the pre-test discussion. Has it all been in vain?

Please extend the test!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-23 23:13:47
According to my schedule, I can only evaluate the results on Saturday, January 14th, 2006, so I am extending the test until Friday, January 13th, 2006.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2005-12-23 23:44:26
perfect
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sehested on 2005-12-24 08:39:46
Thank you
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Oliver on 2005-12-25 05:29:01
Quote
We are slowly approaching the deadline and I still don't have enough results for several samples.
Considering 10 results per sample are minimum, this is what I ask people to focus on :

Code: [Select]
Sample number:                9    13    15    16    17
-------------------------------------------------------
Missing results (at least):   1     2     2     3     3

[a href="index.php?act=findpost&pid=351772"][{POST_SNAPBACK}][/a]

I know the deadline has been extended.  However, it may still be useful to post this info on the first page.

Oliver
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: naylor83 on 2005-12-25 10:34:42
Quote
According to my schedule, I can only evaluate the results on Saturday, January 14th, 2006, so I am extending the test until Friday, January 13th, 2006.
[a href="index.php?act=findpost&pid=352114"][{POST_SNAPBACK}][/a]


Great. I'll send you my first 11 results now, though, since you want them.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-25 10:39:48
Quote
Quote
According to my schedule, I can only evaluate the results on Saturday, January 14th, 2006, so I am extending the test until Friday, January 13th, 2006.
[a href="index.php?act=findpost&pid=352114"][{POST_SNAPBACK}][/a]


Great. I'll send you my first 11 results now, though, since you want them.
[a href="index.php?act=findpost&pid=352273"][{POST_SNAPBACK}][/a]


If you want to test more samples, feel free to post everything at once.

Edit: Too late.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: LANjackal on 2005-12-25 17:28:17
Thanks, Sebastian .
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-26 10:28:15
Two minutes ago, I just received the third unencrypted result. I would like to ask everyone not to set up the test for themeselves, but use the pre-made configuration files (configXX.ecf - XX being the number of the sample you want to test). Using them, the results are automatically encrypted upon saving. Please send only encrypted results.

Thank you.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: MedO on 2005-12-26 14:34:06
I give up for now.  The low anchor is always obvious for me, but I can't tell any other sample from the original.
Don't know if it's my ears, my lack of training or my equipment... well, I'll be back for the 48kbps test.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-26 22:13:10
In case anyone is interested, final bitrate table is available here: http://www.maresweb.de/listening-tests/mf-...ous/bitrate.htm (http://www.maresweb.de/listening-tests/mf-128-1/miscellaneous/bitrate.htm)
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: moozooh on 2005-12-27 23:50:15
Thanks for the extension, Sebastian. I'll try to contribute, it seems I'll have enough time now. This test is pretty hard, I must say…
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: OverlordQ on 2005-12-28 00:21:30
BAH, was hoping to come back from xmas and find the results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2005-12-28 02:33:26
and did you contributed anything?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: zima on 2005-12-28 04:50:18
I'm planning to contribute in this listening test...it'll be my first.
I've read the documentation, however I would be gratefull for answers for questions below (sorry if they were answered previously/somewhere - I have acces to net only in cafe right now...)

As I understand, ABC-HR part will "directly" provide the results, and ABX part is neccesary to...what exactly? Determining how much "weight" should be put on individual results while calculating end results? Or perhaps it's "just" to provide the threshold of 0,05 determining which results should be taken into account and which not, at all?
The reason I'm asking is to know if I should provide the results for all samples (doing ABX between original and each of them until the probability will stabilize somehow). Or perhaps it's entirelly acceptable to first ABX the samples to know which ones give me desirable probability, beacuse these ones will be taken into account while calculating end results and rest - not?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Garf on 2005-12-28 09:10:05
The ABX part is there to help YOU make sure that a "difference" you are hearing really exists.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: zima on 2005-12-29 18:26:35
So...it won't have any effect on end results of whole test?    (of course other than the case when ABX results are "against" ABC/HR ones  )
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: guruboolez on 2005-12-29 18:29:08
Quote
(of course other than the case when ABX results are "against" ABC/HR ones   )
[a href="index.php?act=findpost&pid=353123"][{POST_SNAPBACK}][/a]

It can't happen with schnofler's ABC/HR. If a file is correctly ABXed, the bad slider is automatically frozen
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-31 19:59:25
OK, I have at least 10 results for all samples now.

Happy New Year to everyone; three hours left in Germany.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: skelly831 on 2005-12-31 20:41:03
Quote
Happy New Year to everyone; three hours left in Germany.
[a href="index.php?act=findpost&pid=353658"][{POST_SNAPBACK}][/a]

Congratulations, it's 12:40 PM where I live.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2005-12-31 23:02:28
2006 finally arrived here, too!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: skelly831 on 2005-12-31 23:13:42
Quote
2006 finally arrived here, too!
[a href="index.php?act=findpost&pid=353672"][{POST_SNAPBACK}][/a]

8 hours and 47 minutes to go...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sinnema on 2006-01-01 02:44:22
Quote
OK, I have at least 10 results for all samples now.
[a href="index.php?act=findpost&pid=353658"][{POST_SNAPBACK}][/a]
Is that enough? 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-01 09:58:20
No, but now people don't have to focus on specific samples for which I have less than 10 results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Cutter on 2006-01-06 15:47:45
Hi,

Is it possible to reset an ABX test after having pressed the "Finish" button? I pressed Finish after the 6th try but would like to test more.
Thanks.

PS: are ABX results included in the results of the listening test? How low the probability has to be to disable the bad slider?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-09 11:56:32
All I can say is that ABX results are included in the logs. No idea about the other questions, though.

Anyways, how's testing going, guys? Slowly reaching the second deadline and I can't say I received a lot of new results.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Garf on 2006-01-09 12:11:50
Quote
All I can say is that ABX results are included in the logs. No idea about the other questions, though.

[a href="index.php?act=findpost&pid=355690"][{POST_SNAPBACK}][/a]


The ABX "probability if guessing" that you must reach is set by the person making the test.

So if you don't remember, I'll guess it's the default <0.05
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-09 12:19:28
Quote
Quote
All I can say is that ABX results are included in the logs. No idea about the other questions, though.

[a href="index.php?act=findpost&pid=355690"][{POST_SNAPBACK}][/a]


The ABX "probability if guessing" that you must reach is set by the person making the test.

So if you don't remember, I'll guess it's the default <0.05
[a href="index.php?act=findpost&pid=355697"][{POST_SNAPBACK}][/a]


Ah, then yes, 0.05 is correct.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: schnofler on 2006-01-09 14:58:37
Quote
Is it possible to reset an ABX test after having pressed the "Finish" button?
[a href="index.php?act=findpost&pid=355055"][{POST_SNAPBACK}][/a]


No, it's not possible. Otherwise you could just keep resetting and finishing the test until you get the desired result (pval<0.05) by chance.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Cutter on 2006-01-09 18:28:53
Quote
Quote
Is it possible to reset an ABX test after having pressed the "Finish" button?
[a href="index.php?act=findpost&pid=355055"][{POST_SNAPBACK}][/a]


No, it's not possible. Otherwise you could just keep resetting and finishing the test until you get the desired result (pval<0.05) by chance.
[a href="index.php?act=findpost&pid=355746"][{POST_SNAPBACK}][/a]

Isn't such cheating already possible by "forgetting" to save the session, re-opening the test and trying again?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Garf on 2006-01-09 18:35:17
Quote
Quote
Quote
Is it possible to reset an ABX test after having pressed the "Finish" button?
[a href="index.php?act=findpost&pid=355055"][{POST_SNAPBACK}][/a]


No, it's not possible. Otherwise you could just keep resetting and finishing the test until you get the desired result (pval<0.05) by chance.
[a href="index.php?act=findpost&pid=355746"][{POST_SNAPBACK}][/a]

Isn't such cheating already possible by "forgetting" to save the session, re-opening the test and trying again?
[a href="index.php?act=findpost&pid=355799"][{POST_SNAPBACK}][/a]


The likelyhood of being able to get a significant ABX on *all* samples with this method isn't very good
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2006-01-12 17:43:47
ok, started testing, some doubts:

1) can i use a pair of sony u$s 10 headphones? (i have no better)
2) as far as i can see, i rate all 5! (i must be deaf) should i try anyway?
3) when i press "ref" button, is it allways the unencoded sample?

thanks.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: sTisTi on 2006-01-12 18:04:37
Quote
ok, started testing, some doubts:

1) can i use a pair of sony u$s 10 headphones? (i have no better)
2) as far as i can see, i rate all 5! (i must be deaf) should i try anyway?
3) when i press "ref" button, is it allways the unencoded sample?

thanks.
[a href="index.php?act=findpost&pid=356575"][{POST_SNAPBACK}][/a]

1) Probably yes if you still can spot artefacts, but it certainly won't make testing any easier for you
2) Same here, except for one sample, and that with Sennheiser HD600's 
But you do recognise the low anchor, don't you? Otherwise you really should consider having your ears checked 
3) Yes
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2006-01-12 18:15:42
Quote
But you do recognise the low anchor, don't you? Otherwise you really should consider having your ears checked 

cant even recognize the f*#$g low anchor

mmm ... so maybe for me 128kbps = CD quality is true!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: IgorC on 2006-01-12 18:22:47
Quote
Quote
But you do recognise the low anchor, don't you? Otherwise you really should consider having your ears checked 

cant even recognize the f*#$g low anchor
[a href="index.php?act=findpost&pid=356585"][{POST_SNAPBACK}][/a]


I can ABXed low anchor  in the almost all samples. But it's hard for  anothers 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2006-01-12 18:26:11
so the question still remain, should i send an all 5 test result?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-12 18:40:27
Well, yes, why not? It IS a result showing that the encoders performed well.

Anyways, no offense, but you could have checked if you can spot differences before telling me to extend the test.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2006-01-12 18:59:52
Quote
Anyways, no offense, but you could have checked if you can spot differences before telling me to extend the test.

never thought it could be this dificult ... maybe i should had tried ... but it bet you didn't do it just for me 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-12 19:32:25
Well, as I said in a previous post, I didn't receive a significant amount of new results which makes me feel a bit upset. It's not your fault, don't get me wrong.

Oh, by the way, thanks a lot to all of you who participated and helped me out during these weeks!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: slav!x on 2006-01-12 19:39:00
Quote
Quote
But you do recognise the low anchor, don't you? Otherwise you really should consider having your ears checked 

cant even recognize the f*#$g low anchor

mmm ... so maybe for me 128kbps = CD quality is true!
[a href="index.php?act=findpost&pid=356585"][{POST_SNAPBACK}][/a]

check sample #12 IMHO quite easy to detect low anchor ...  but other encoders for me mission impossible  and sample #1 annoying in original  haven't checked others.. anyway can't recognise anything except low anchor...
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2006-01-12 20:10:46
i would check sample 12 right away. i suppose my cheap earphones could be the culprid also?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: STL on 2006-01-12 20:25:26
Quote
Well, as I said in a previous post, I didn't receive a significant amount of new results which makes me feel a bit upset.
Well I just attempted to try to help with that, but I don't hear any sound when I try to play the a sample.  In fact, I get no indication they're actually being played (is that normal)?  I tried listen to just part of the sample with the same result.  I believe I followed all the directions.  When I play the WAVs with Windows Media Player I hear them.  I upgrade to the lastest java (as I had an old version) but also didn't help.  I also verified my directory structure is correct.  Any ideas?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: STL on 2006-01-12 20:27:53
I figured it out.  I had to change my playback device setting.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: kwanbis on 2006-01-12 20:28:02
options -> settings -> playback, and set it to "Primary Sound Driver".

edit: cool.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Pio2001 on 2006-01-12 23:34:05
Quote
i would check sample 12 right away. i suppose my cheap earphones could be the culprid also?
[a href="index.php?act=findpost&pid=356611"][{POST_SNAPBACK}][/a]


You should not send the results of sample 12, unless you send the results for every other sample also.
Choosing sample 12 on someone else's advice will bias your answers.

Even if you send every result, it will be somewhat biased, because you would have paid more attention to sample 12 than to the other samples, bit this problem is less important.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Cutter on 2006-01-13 05:02:20
Quote
Well, as I said in a previous post, I didn't receive a significant amount of new results which makes me feel a bit upset. It's not your fault, don't get me wrong.

In that case why not extending the test until you get enough results? You're not in a hurry, are you?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-13 06:03:42
Quote
Quote
Well, as I said in a previous post, I didn't receive a significant amount of new results which makes me feel a bit upset. It's not your fault, don't get me wrong.

In that case why not extending the test until you get enough results? You're not in a hurry, are you?
[a href="index.php?act=findpost&pid=356715"][{POST_SNAPBACK}][/a]


Because people really had enough time now. If they didn't test now, they won't test either.
I have to admit that the test is really hard and the actual bitrate for the encoded material is not 128 kbps, but 140 kbps. I think this makes people step back.

Anyways, the test will end today, 23:59:59 GMT.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Synthetic Soul on 2006-01-13 07:34:28
I seem to remember reading about the Playback Device.

Try selecting "Options" > "Settings" > "Playback" and changing the selected device in the drop down.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: matthiasb on 2006-01-13 07:42:01
Quote
Anyways, the test will end today, 23:59:99 GMT.
23:59:59?
If it is, i will be one of the first reading it.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Serge Smirnoff on 2006-01-13 07:46:42
Quote
Anyways, the test will end today, 23:59:99 GMT.
[a href="index.php?act=findpost&pid=356723"][{POST_SNAPBACK}][/a]
Does this mean that we'll see the results at 24:00:00?
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-13 11:32:28
Quote
Quote
Anyways, the test will end today, 23:59:99 GMT.
23:59:59?
If it is, i will be one of the first reading it.
[a href="index.php?act=findpost&pid=356732"][{POST_SNAPBACK}][/a]


Oops, typo!


Quote
Quote
Anyways, the test will end today, 23:59:99 GMT.
[a href="index.php?act=findpost&pid=356723"][{POST_SNAPBACK}][/a]
Does this mean that we'll see the results at 24:00:00?
[a href="index.php?act=findpost&pid=356733"][{POST_SNAPBACK}][/a]


Nope. What I wanted to say is that no results posted after that date are going to be accepted. I will do my best to post the results on Saturday afternoon.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: eisa01 on 2006-01-13 11:37:06
Looking forward to the results, it was very interesting for me to participate in this test. Didn't know my hearing was so bad
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-13 23:00:05
One hour left...

BTW, results are most likely going to be published on Saturday around 2 PM GMT. Hope you understand that I am pretty busy with school.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: minisu on 2006-01-13 23:55:04
Quote
One hour left...

BTW, results are most likely going to be published on Saturday around 2 PM GMT. Hope you understand that I am pretty busy with school.
[a href="index.php?act=findpost&pid=356878"][{POST_SNAPBACK}][/a]

No problem. You've done a great job and it's a pity that so few did the test.

Still, I can't wait to see the results 
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Caroliano on 2006-01-14 00:09:38
I was late by 2min! Wondering what is the tolerance.... I tested 6 samples (12-18). Found some diferences between the main codecs (excluding the low-anchor) in two or three samples IIRC.

The longer time to test helped a bit: I tested 2~3 samples more than I would if the test ended in 25th. But after the extending I forgot about the test, and remembered only two days ago. That is the side efect.

Anyway: thanks for this oportunity!
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-14 09:07:59
Quote
I was late by 2min! Wondering what is the tolerance.... I tested 6 samples (12-18). Found some diferences between the main codecs (excluding the low-anchor) in two or three samples IIRC.

The longer time to test helped a bit: I tested 2~3 samples more than I would if the test ended in 25th. But after the extending I forgot about the test, and remembered only two days ago. That is the side efect.

Anyway: thanks for this oportunity!
[a href="index.php?act=findpost&pid=356888"][{POST_SNAPBACK}][/a]


According to the mail header, one minute too late. It's OK, I am not that anal about it.  Thanks for submitting.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-14 16:04:15
OK, finished school work. I now started working on the results and hope to be able to post them within the next one or two hours. Stay tuned.

I think this thread can be closed now.
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Garf on 2006-01-14 16:59:45
Maybe it's better to close only after the results thread is up?


(Yes, I'm getting more careful with that now )
Title: Multiformat Listening Test @ 128 kbps - OPEN
Post by: Sebastian Mares on 2006-01-15 13:04:48
Results are available here (http://www.hydrogenaudio.org/forums/index.php?showtopic=40607). Thread can safely be closed now.