Skip to main content

Topic: 64 kbps listening test 2005 (Read 81330 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • sehested
  • [*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #50
Yet another sample suggestion:

Dire Straits - Iron Hand

  • S_O
  • [*][*][*][*]
64 kbps listening test 2005
Reply #51
Quote
The HE-AAC encoder from Helix Producer hasn't been improve since last 64kbps listening tests. There is only new thing : introduction of new target bitrates (32 & 48 kbps).
Huh? I cannot remember a listening test with Helix HE-AAC being tested. There were a 64kbps test with RealAudio Cook once, but with Helix HE-AAC? I cannot remember.

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #52
Quote
But if you need to run a separate test anyway then you might as well do it beforehand.  Maybe there are serious flaws which could ruin Vorbis' reputation![a href="index.php?act=findpost&pid=284685"][{POST_SNAPBACK}][/a]


Yes, I believe it would be very bad if Vorbis reputation got ruined thanks to a badly tuned encoder.

Also, this test is only supposed to start after Apple releases iTunes 5 / QuickTime 7. That should give people plenty of time to conduce paralel tests, if there is really any interest...

Quote
The HE-AAC encoder from Helix Producer hasn't been improve since last 64kbps listening tests.
[a href="index.php?act=findpost&pid=284689"][{POST_SNAPBACK}][/a]


It wasn't even tested back then :B
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

64 kbps listening test 2005
Reply #53
IMO this sample could be very useful for this test. In particular he cause a lot of phasing problems with lame --preset cbr 128 and Vorbis low bitrates.

Download HERE

  • znode
  • [*][*]
64 kbps listening test 2005
Reply #54
Quote
But if you need to run a separate test anyway then you might as well do it beforehand.  Maybe there are serious flaws which could ruin Vorbis' reputation!
[{POST_SNAPBACK}][/a]


Yeah, misinterpretation of results is quite a problem. I've seen countless people claim vorbis to be always better than any other codec, in all cases, because of those [a href="http://www.rjamorim.com/test/multiformat128/results.html]http://www.rjamorim.com/test/multiformat128/results.html[/url] results at 128kbps.

  • HotshotGG
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #55
Quote
If wma would be left outside, half of wannabee slashdotters would be asking where is wma, the cd quality at 64 kbps codec. Were the HA Lame and mpc lovers afraid of it?


  I don't know what to make of that site so I don't ask. Informative yes. Always on par with issues at hand no. Rating system nah    For a site full of edcuated nerds they sure don't act like it sometimes  .

Quote
IMO this sample could be very useful for this test. In particular he cause a lot of phasing problems with lame --preset cbr 128 and Vorbis low bitrates.


channel coupling related maybe? hmm I will have to test that out myself after.

Quote
Yeah, misinterpretation of results is quite a problem. I've seen countless people claim vorbis to be always better than any other codec, in all cases, because of those tests


Well it would be great two see both Nero HE-AAC and Vorbis tied for first ;-D.  A streaming listening test was definitely going to be needed though eventually.
budding I.T professional

  • kurtnoise
  • [*][*][*][*]
64 kbps listening test 2005
Reply #56
Quote
Quote
The HE-AAC encoder from Helix Producer hasn't been improve since last 64kbps listening tests.
[a href="index.php?act=findpost&pid=284689"][{POST_SNAPBACK}][/a]


It wasn't even tested back then :B
[a href="index.php?act=findpost&pid=284708"][{POST_SNAPBACK}][/a]

ouups, sorry for the confusion...  I was tired last night.



So, He-AAC encoder from Producer could be interesting though...

64 kbps listening test 2005
Reply #57
Quote
Sample proposition: the beginning of "Money" by Pink Floyd.

I do not have it available, but I am sure some Pink Floyd fan could upload it.
Basically it is background music with coins and cash machine sounds. I think that the coins coud be interesting.
[{POST_SNAPBACK}][/a]


[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=32628]http://www.hydrogenaudio.org/forums/index....showtopic=32628[/url]

Anyone interested in Time?
  • Last Edit: 23 March, 2005, 04:45:53 AM by Sebastian Mares

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #58
Why not taking also 3gpp's CT HE + PS AAC?

64 kbps listening test 2005
Reply #59
Quote
Why not taking also 3gpp's CT HE + PS AAC?
[a href="index.php?act=findpost&pid=284794"][{POST_SNAPBACK}][/a]


Wouldn't that be a HE-AAC 64 kbps listening test then?

  • Gabriel
  • [*][*][*][*][*]
  • Developer
64 kbps listening test 2005
Reply #60
All those propositions are transforming this into a 64kbps MPEG test...

64 kbps listening test 2005
Reply #61
Anyways...

Regarding Vorbis, I would love if some Vorbis users could start a small listening test and compare AoTuV3 and Xiph 1.1 so that the better version will be used in this test.

As for the discussion about mp3PRO or ATRAC3+, I think that I will use ATRAC3+ since it is more wide-spread than mp3PRO and since mp3PRO didn't change since the last test.

Still not sure what to do with WMA - either Standard or Professional. I, personally, would choose Standard since it's the format you find in music stores and it's also what most people use so it's compatible with their players.

  • moi
  • [*][*]
64 kbps listening test 2005
Reply #62
I think you really need to include WMA Standard, as it is probably the most common format encoded at 64kbps, people will want to see how it compares with others in the test. Of course the newest, WMA 9.1, which is installed with WMP10.

You might also wish to include WMA Pro, to see how it compares at that bit rate to standard.

I really don't see why LAME at 128kbps should be  included in a 64kbps listening test, as it was the other time. Probably has something to do with the claim that WMA at 64kbps sounds "as good as" MP3 at 128kbps. I don't think many here believe that claim. In any case, IMO, a 64kbps listening test should only include music encoded at 64kbps. It is misleading to encode 128kbps in one format, and 64kbps in all the others. Everything in a 64kbps listening test should be encoded at 64kbps.

I think MP3Pro should be included, as it did very well on some 64kbps tests in the past. Not supported by many players, but by some, I think it should be included, whether or not it changed.

  • Redmond
  • [*]
64 kbps listening test 2005
Reply #63
Quote
You might also wish to include WMA Pro, to see how it compares at that bit rate to standard.
[a href="index.php?act=findpost&pid=284822"][{POST_SNAPBACK}][/a]


"Publicly available" WMA Pro encoders do not go down to 64Kb/s stereo.

  • Latexxx
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #64
Quote
I really don't see why LAME at 128kbps should be  included in a 64kbps listening test, as it was the other time. Probably has something to do with the claim that WMA at 64kbps sounds "as good as" MP3 at 128kbps. I don't think many here believe that claim. In any case, IMO, a 64kbps listening test should only include music encoded at 64kbps. It is misleading to encode 128kbps in one format, and 64kbps in all the others. Everything in a 64kbps listening test should be encoded at 64kbps.
[a href="index.php?act=findpost&pid=284822"][{POST_SNAPBACK}][/a]

A credible listening test should have a low and high anchor.

  • Aoyumi
  • [*][*][*]
64 kbps listening test 2005
Reply #65
Quote
Regarding Vorbis, I would love if some Vorbis users could start a small listening test and compare AoTuV3 and Xiph 1.1 so that the better version will be used in this test.
[a href="index.php?act=findpost&pid=284811"][{POST_SNAPBACK}][/a]

When the test is performed, I need to submit the newest experiment version.
It is more clearly than aoTuV beta3 good with some samples (setting to the low bit rate).

  • PoisonDan
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #66
Quote
Quote
Regarding Vorbis, I would love if some Vorbis users could start a small listening test and compare AoTuV3 and Xiph 1.1 so that the better version will be used in this test.
[a href="index.php?act=findpost&pid=284811"][{POST_SNAPBACK}][/a]

When the test is performed, I need to submit the newest experiment version.
It is more clearly than aoTuV beta3 good with some samples (setting to the low bit rate).
[a href="index.php?act=findpost&pid=284836"][{POST_SNAPBACK}][/a]

Were you planning on releasing a new version soon anyway? I wouldn't want you to feel rushed to get a version out the door just to be in time for this listening test...

At this moment, I'm extremely busy with real-life and work-related stuff, but next week I'll probably have some time to do a few Vorbis listening tests...
Over thinking, over analyzing separates the body from the mind.

64 kbps listening test 2005
Reply #67
Well, take your time, since the test will start after Apple releases their HE-AAC encoder.

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #68
Quote
"Publicly available" WMA Pro encoders do not go down to 64Kb/s stereo.
[a href="index.php?act=findpost&pid=284828"][{POST_SNAPBACK}][/a]


If I remember correctly, the publicly available encoder stays around 64kbps if you choose the lowest VBR setting (10).
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

64 kbps listening test 2005
Reply #69
Regarding the low anchor, do you think LAME or FhG should be used at 64 kbps?

  • sehested
  • [*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #70
Quote
Regarding the low anchor, do you think LAME or FhG should be used at 64 kbps?
[a href="index.php?act=findpost&pid=284976"][{POST_SNAPBACK}][/a]

I would like to see LAME as low anchor.

That would also demonstrate the improvements of the other codecs compared to the best MP3 encoder available.

  • ff123
  • [*][*][*][*][*]
  • Developer (Donating)
64 kbps listening test 2005
Reply #71
Quote
Quote
Regarding the low anchor, do you think LAME or FhG should be used at 64 kbps?
[a href="index.php?act=findpost&pid=284976"][{POST_SNAPBACK}][/a]

I would like to see LAME as low anchor.

That would also demonstrate the improvements of the other codecs compared to the best MP3 encoder available.
[a href="index.php?act=findpost&pid=284995"][{POST_SNAPBACK}][/a]


This presumes that lame is the best mp3 encoder at 64 kbps, which isn't a given.  The question of which mp3 encoder to use as a low anchor probably deserves a pre-test if people are interested in using the best-sounding one.

ff123

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #72
Two suggestions:

• I think that the current scale isn't really really suited to a 64 kbps and the expected distortions.
Artifacts “perceptible but not annoying” (4.0) are maybe not very common at this bitrate. And few encoders are able to reproduce (in my opinion) a sound with only "slightly annoying" (3.0) difference at 64 kbps. It's possible to change the corresponding scale with schnofler's abc/hr, and I wonder if it's not worth to think about it. If I remember correctly, the average notation I gave to most encoders during the 32 kbps was inferior to 1.5/5 


• A also suggest to reduce the length of all samples. I'm the first one to provide 30 seconds samples, but I perfectly know the drawbacks. Some people will rate one encoder on a short range located at the beginning, some other will evaluate another part (totally different from the first one), etc... Finally it's exactly if people have evaluate different samples. I suggest to limit the duration to 6 or 7 seconds.

64 kbps listening test 2005
Reply #73
Quote
• A also suggest to reduce the length of all samples. I'm the first one to provide 30 seconds samples, but I perfectly know the drawbacks. Some people will rate one encoder on a short range located at the beginning, some other will evaluate another part (totally different from the first one), etc... Finally it's exactly if people have evaluate different samples. I suggest to limit the duration to 6 or 7 seconds.
[a href="index.php?act=findpost&pid=285081"][{POST_SNAPBACK}][/a]


I understand what you mean, but why not let testers decide which portion they want to ABX?
  • Last Edit: 24 March, 2005, 07:09:58 AM by Sebastian Mares

  • Gabriel
  • [*][*][*][*][*]
  • Developer
64 kbps listening test 2005
Reply #74
I also think that 30s might be too long.
Perhaps 6s is too short, but I think that 15s should be enough.

Letting testers deciding which portion to use is perhaps reducing "usefullness" of results. It is like they are testing different samples, but it makes correlation between results for the same sample harder.

If a sample has some quite different parts in a 30s set, then it could be intersting to split it into 2 samples, making interpretation of results easier.