Skip to main content

Topic: 64 kbps listening test 2005 (Read 81293 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
64 kbps listening test 2005
Greetings!

I am planning to conduce a multi-format 64 kbps listening test under Roberto's supervision. The exact date for the beginning of the test is not set, since I am waiting for Apple to release their HE-AAC encoder sometime in April (according to the rumours).

So far, here is what I was thinking about. Feedback and suggestions are welcome.

Codecs:
  • Nero HE-AAC
  • Apple HE-AAC
  • Ogg Vorbis
  • Windows Media Audio
  • LAME 128 kbps as high anchor
  • Fraunhofer 64 kbps as low anchor
  • mp3PRO maybe
Nero and Apple HE-AAC will be included for sure.
What I don't really know is which Vorbis version to use - either AoTuV3 or 1.1. Same applies to WMA - either Standard due to compatibility or Professional for quality. Last but not least, should mp3PRO be tested again as reference (since there aren't any changes from version used in Roberto's last test)?

As you can see, there will be 6 or 7 (depending on mp3PRO) codecs.

Samples:
  • DaFunk
  • EnolaGay
  • experienca
  • gone
  • Illinois
  • mybloodrusts
  • NewYorkCity
  • riteofspring
  • Scars
I am planning to use 18 samples - 9 from Roberto's old listening test and 9 additional ones. What do you guys suggest?

The results will be calculated in the same way Roberto's results were calculated.

So, what are you thoughts? 

Regards,
Sebastian

Edit: Spelling and typo in the number of samples planned to use.
  • Last Edit: 22 March, 2005, 10:00:27 AM by Sebastian Mares

64 kbps listening test 2005
Reply #1
As difficult sample, I would suggest Kraftwerk to replace Waiting (which was used in the last test and is not missing).

  • sehested
  • [*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #2
Good initiative. I will look forward to the test.

I suggest to leave mp3PRO out, since nobody use it.

As for WMA I would recommend using WMA std. It is more widespread than PRO and is what most people refer to as WMA. People already have a hard time distinguishing between std. and PRO so choosing PRO is likely to lead to misinterpretation of the test results. As an alternative one could consider including both WMA std and WMA PRO, anyway my vote on WMA std.

What about using LAME 3.97a8 as low anchor?

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #3
I would take Sony's Atrac3plus @ 64 kbps. It has shown some potential with non-problem samples and it would be great to compare it to other formats.

  • PaleGreen
  • [*]
64 kbps listening test 2005
Reply #4
I'd like to see both WMA Standard & Pro included.

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #5
Quote
I'd like to see both WMA Standard & Pro included.
[a href="index.php?act=findpost&pid=284530"][{POST_SNAPBACK}][/a]

I wouldn't include wma pro because no internet radio uses it and it would only confuse people.

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #6
Quote
Quote
I'd like to see both WMA Standard & Pro included.
[a href="index.php?act=findpost&pid=284530"][{POST_SNAPBACK}][/a]

I wouldn't include wma pro because no internet radio uses it and it would only confuse people.
[a href="index.php?act=findpost&pid=284533"][{POST_SNAPBACK}][/a]


I wouldn't include WMA standard because it has already been tested and, as far as I know, it hasn't changed much, if any, since
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • sehested
  • [*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #7
Quote
I wouldn't include WMA standard because it has already been tested and, as far as I know, it hasn't changed much, if any, since
[a href="index.php?act=findpost&pid=284537"][{POST_SNAPBACK}][/a]


May I then suggest we use it as reference?

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #8
Quote
May I then suggest we use it as reference?
[a href="index.php?act=findpost&pid=284539"][{POST_SNAPBACK}][/a]


FhG 64kbps (low anchor) didn't change much either, AFAIK.
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • sehested
  • [*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #9
Quote
Quote
May I then suggest we use it as reference?
[a href="index.php?act=findpost&pid=284539"][{POST_SNAPBACK}][/a]


FhG 64kbps (low anchor) didn't change much either, AFAIK.
[a href="index.php?act=findpost&pid=284541"][{POST_SNAPBACK}][/a]

In that case we can leave out WMA completely

  • Gabriel
  • [*][*][*][*][*]
  • Developer
64 kbps listening test 2005
Reply #10
Perhaps Lame could be used as low anchor?

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #11
Quote
Quote
Quote
May I then suggest we use it as reference?
[a href="index.php?act=findpost&pid=284539"][{POST_SNAPBACK}][/a]


FhG 64kbps (low anchor) didn't change much either, AFAIK.
[a href="index.php?act=findpost&pid=284541"][{POST_SNAPBACK}][/a]

In that case we can leave out WMA completely
[a href="index.php?act=findpost&pid=284544"][{POST_SNAPBACK}][/a]

If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?
  • Last Edit: 22 March, 2005, 10:49:33 AM by Latexxx

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #12
Quote
I would take Sony's Atrac3plus @ 64 kbps. It has shown some potential with non-problem samples and it would be great to compare it to other formats.
[{POST_SNAPBACK}][/a]

I second that suggestion. Atrac3plus wasn't tested yet (only atrac3@132 kbps). It could be very interesting to see how will perform this format, which correspond (according to optimistic estimations made by Sony) to atrac@132 and to atrac@292 kbps:

[a href="http://www.mdfr.com/faqs/atrac_roadmap_small.jpg]http://www.mdfr.com/faqs/atrac_roadmap_small.jpg[/url]

But it's necessary to upload them as lossless files (flac or something else).

  • Thoifl
  • [*]
64 kbps listening test 2005
Reply #13
Quote
What I don't really know is which Vorbis version to use - either AoTuV3

I would take AoTuV3, since it should be tweaked for low bitrates and is little testet.

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #14
Quote
If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?
[{POST_SNAPBACK}][/a]


Good point. But then again, most slashdotters can go .... themselves, for all that I really care...

Quote
Quote
I would take Sony's Atrac3plus @ 64 kbps. It has shown some potential with non-problem samples and it would be great to compare it to other formats.
[a href="index.php?act=findpost&pid=284529"][{POST_SNAPBACK}][/a]

I second that suggestion. Atrac3plus wasn't tested yet (only atrac3@132 kbps). It could be very interesting to see how will perform this format, which correspond (according to optimistic estimations made by Sony) to atrac@132 and to atrac@292 kbps:

[a href="http://www.mdfr.com/faqs/atrac_roadmap_small.jpg]http://www.mdfr.com/faqs/atrac_roadmap_small.jpg[/url]

But it's necessary to upload them as lossless files (flac or something else).
[a href="index.php?act=findpost&pid=284551"][{POST_SNAPBACK}][/a]


Well, same would have to happen with MP3pro, that is another alternative for 7th codec.

So, I guess the discussion is about of wether Atrac3+ or MP3pro should be tested.

I already feel sorry for Sebastian if people choose Atrac3+ 
  • Last Edit: 22 March, 2005, 10:58:53 AM by rjamorim
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • DreamTactix291
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #15
Quote
What I don't really know is which Vorbis version to use - either AoTuV3 or 1.1. Same applies to WMA - either Standard due to compatibility or Professional for quality.
Since aoTuV b2 (which 1.1 is essentially) has been fairly well tested it might be interesting to see how the relative newcomer aoTuV b3 fares IMO.  Even worse possible scenario of regression I can't see it being really that much worse than 1.1 (which I doubt it is worse but only testing can prove either way).

As far as WMA I'd go standard.  Sure it's not as high quality as pro but I've never even come across a WMA Pro file I didn't make myself in the wild.
Nero AAC 1.5.1.0: -q0.45

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #16
Quote
Perhaps Lame could be used as low anchor?
[a href="index.php?act=findpost&pid=284545"][{POST_SNAPBACK}][/a]


Dunno... wouldn't FhG be more recommended, since it is, at least in theory, better at low bitrates thanks to IS?
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • sTisTi
  • [*][*][*][*]
64 kbps listening test 2005
Reply #17
Quote
If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?
[a href="index.php?act=findpost&pid=284549"][{POST_SNAPBACK}][/a]

WMA std should definitely be included. I'm sure it will fail badly against Vorbis and HE AAC, so this test will be one more evidence against Microsoft's ridiculous claims and demonstrate the low quality of WMA compared to other modern codecs.
Proverb for Paranoids: "If they can get you asking the wrong questions, they don't have to worry about answers."
-T. Pynchon (Gravity's Rainbow)

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #18
atrac3plus could be tested over mp3pro, for some reasons:
- mp3pro was tested previously, and hasn't progress since.
- atrac3plus wasn't tested
- there are more device playing atrac3plus than mp3pro (all modern HiMD units, many CD players, Jukebox and Sony flash USB devices).
- atrac3plus have more supporters (I'd use another word to describe them...) than mp3pro, which has progressively lost importance with time and competition.

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #19
Quote
I already feel sorry for Sebastian if people choose Atrac3+ 
[a href="index.php?act=findpost&pid=284557"][{POST_SNAPBACK}][/a]

If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #20
Quote
Quote
Perhaps Lame could be used as low anchor?
[a href="index.php?act=findpost&pid=284545"][{POST_SNAPBACK}][/a]


Dunno... wouldn't FhG be more recommended, since it is, at least in theory, better at low bitrates thanks to IS?
[a href="index.php?act=findpost&pid=284559"][{POST_SNAPBACK}][/a]

MP3 at 64 kbps must be a low anchor. It doesn't matter to use lame or fhg. All we need is a encoder which sound worse than all other competitors.
Testing lame at 128 and at 64 kbps is IMO a good idea.

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #21
Quote
Quote
I already feel sorry for Sebastian if people choose Atrac3+ 
[a href="index.php?act=findpost&pid=284557"][{POST_SNAPBACK}][/a]

If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
[a href="index.php?act=findpost&pid=284564"][{POST_SNAPBACK}][/a]

I must warn you that basic MD lovers won't consider results as significant unless you've made encoding directly from device, and not from SonicStage. Hardware encoding is often considered as better than software encodings (placebo or not: I don't know — blind listening tests were never done to my knowledge).

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #22
Quote
If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
[a href="index.php?act=findpost&pid=284564"][{POST_SNAPBACK}][/a]


Ah, that's good news, indeed. But he'll still have to install SonicStage. That flaming piece of poo ruined my win2000 installation.

Quote
I must warn you that basic MD lovers won't consider results as significant unless you've made encoding directly from device, and not from SonicStage. Hardware encoding is often considered as better than software encodings (placebo or not: I don't know — blind listening tests were never done to my knowledge).


That is certifiable bollocks. Hardware encoding has no reason to be better than software encoding, quite the opposite: hardware encoding faces limitations related to battery consumption, real-time processing on underpowered DSPs, integer-only processing, etc, etc.
  • Last Edit: 22 March, 2005, 11:06:35 AM by rjamorim
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • Gabriel
  • [*][*][*][*][*]
  • Developer
64 kbps listening test 2005
Reply #23
Quote
Dunno... wouldn't FhG be more recommended, since it is, at least in theory, better at low bitrates thanks to IS?

I am not sure if the difference would be that big between FhG and 3.97b.

Anyway, whatever the choosen anchors, I'd like the graphs of results mentionning "low anchor" and "high anchor" instead of the codec name.

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #24
Quote
Quote
If you mean preparing samples, it is pretty easy nowadays using a program called himdrenderer.
[a href="index.php?act=findpost&pid=284564"][{POST_SNAPBACK}][/a]


Ah, that's good news, indeed. But he'll still have to install SonicStage. That flaming piece of poo ruined my win2000 installation.

That is certifiable bollocks. Hardware encoding has no reason to be better than software encoding, quite the opposite: hardware encoding faces limitations related to battery consumption, real-time processing on underpowered DSPs, integer-only processing, etc, etc.
[a href="index.php?act=findpost&pid=284568"][{POST_SNAPBACK}][/a]


I have a working SS installation if that matters.

There are some hardware atrac3 vs SS atrac3 abx threads somewhere.