Skip to main content

Topic: 64 kbps listening test 2005 (Read 81174 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
64 kbps listening test 2005
Reply #100
Quote
As I said, I was not fully satisfied by this test (too fast, too imprecise). If the collective test doesn't start in the next days, I think I could test CBR and VBR again, without WMApro this time, and with ABX phase in order to be sure that difference were audible.
[a href="index.php?act=findpost&pid=287014"][{POST_SNAPBACK}][/a]


Well, I guess it will take at least one more week for the test to start. Apple's HE-AAC encoder isn't even out yet.

Also, again, Vorbis tests are welcome to help deciding which encoder to use.

  • Busemann
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #101
Quote
Well, I guess it will take at least one more week for the test to start. Apple's HE-AAC encoder isn't even out yet.
[a href="index.php?act=findpost&pid=287022"][{POST_SNAPBACK}][/a]



I reckon HE-AAC is still a bit off, definitely more than 1 week.

  • slippyC
  • [*][*]
64 kbps listening test 2005
Reply #102
I know this is a little late in the thread and I may have missed it, but I think you ought to do both WMA and WMAPro.

Reason being is because WMA is widespread, but WMAPro is superior to Standard.  It would keep people from becoming confused and still giving the WMAPro codec a fair chance(to keep the WMA fanatics off your back).

  • rjamorim
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #103
Quote
I reckon HE-AAC is still a bit off, definitely more than 1 week.
[a href="index.php?act=findpost&pid=287032"][{POST_SNAPBACK}][/a]


Rumour has it that Tiger will be released mid-april. Of course, Apple rumours are NEVER trustworthy.

Quote
Reason being is because WMA is widespread, but WMAPro is superior to Standard.  It would keep people from becoming confused and still giving the WMAPro codec a fair chance(to keep the WMA fanatics off your back).
[a href="index.php?act=findpost&pid=287048"][{POST_SNAPBACK}][/a]


I think the level of usefulness of WMA Pro at 64kbps is too small. Microsoft clearly sends you the message that they only want you using it above 128kbps (in CBR mode at least). Forcing it to go lower with VBR results in too random bitrates that would make the test biased for one side or the other.

Also, it's worth remembering such low bitrates are useful for a) limited memory hardware or b) streaming. a) is pointless because no portable player supports WMA Pro. b) makes no sense because VBR doesn't mix well with streaming.
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • slippyC
  • [*][*]
64 kbps listening test 2005
Reply #104
Quote
I think the level of usefulness of WMA Pro at 64kbps is too small. Microsoft clearly sends you the message that they only want you using it above 128kbps (in CBR mode at least). Forcing it to go lower with VBR results in too random bitrates that would make the test biased for one side or the other.

Also, it's worth remembering such low bitrates are useful for a) limited memory hardware or b) streaming. a) is pointless because no portable player supports WMA Pro. b) makes no sense because VBR doesn't mix well with streaming.
[a href="index.php?act=findpost&pid=287061"][{POST_SNAPBACK}][/a]



Yea, I don't know of any portable support.  If I remember right someone on this board said that MS never intended it to be.

Anyway, was just a suggestion for some folks concerns.  I don't use WMAPro, have messed with it a little in the past, so didn't matter much to me.

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #105
Suggestion: if you plan to publish a bitrate table, and in order to avoid (at least to limit) the usual discussion about 'fairness', 'apples & orange', etc... why not publishing a bitrate table of the full encoded song instead of the short samples (~20 sec), which are probably higher.

It's not really easy (you have to download full samples, or have to ask to people submitting samples to encode their files for you and then report the exact value), but it could be more representative of the bitrate of very short samples. On visual tests (video, cf. doom9.org), we have the bitrate for the full movie, and not for the selected frames. What do you think?

  • Gabriel
  • [*][*][*][*][*]
  • Developer
64 kbps listening test 2005
Reply #106
Regarding the low anchor, I would personnally like Lame @abr.
My own justification is:
*that would probably not change that much the notation of the low anchor, and anyway the purpose of the low anchor is to be low.
*that would allow me to have fine input regarding Lame's true performance @64k compared to modern codecs

  • AtaqueEG
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #107
Quote
Suggestion: if you plan to publish a bitrate table, and in order to avoid (at least to limit) the usual discussion about 'fairness', 'apples & orange', etc... why not publishing a bitrate table of the full encoded song instead of the short samples (~20 sec), which are probably higher.

It's not really easy (you have to download full samples, or have to ask to people submitting samples to encode their files for you and then report the exact value), but it could be more representative of the bitrate of very short samples. On visual tests (video, cf. doom9.org), we have the bitrate for the full movie, and not for the selected frames. What do you think?
[a href="index.php?act=findpost&pid=287162"][{POST_SNAPBACK}][/a]


This is a great idea.
And it would give you a "real life" scenario, showing the actual behavior of the codec with full songs, and how it would act on such tracks.
I'm the one in the picture, sitting on a giant cabbage in Mexico, circa 1978.
Reseñas de Rock en Español: www.estadogeneral.com

64 kbps listening test 2005
Reply #108
Quote
• I think that the current scale isn't really really suited to a 64 kbps and the expected distortions.
Artifacts “perceptible but not annoying” (4.0) are maybe not very common at this bitrate. And few encoders are able to reproduce (in my opinion) a sound with only "slightly annoying" (3.0) difference at 64 kbps. It's possible to change the corresponding scale with schnofler's abc/hr, and I wonder if it's not worth to think about it. If I remember correctly, the average notation I gave to most encoders during the 32 kbps was inferior to 1.5/5 

I second that suggestion. During the 32 kbps test, I gave most sample only between 1 and 2 because then I decided to follow the comments near the scale while (it seems that) a lot of people use the whole scale without taking into account the comments '“perceptible but not annoying” and so on...).

I mean that the situation should be clarified: should we use the whole scale or follow the comments?
  • Last Edit: 31 March, 2005, 10:27:11 AM by nyarlathotep

64 kbps listening test 2005
Reply #109
Quote
I mean that the situation should be clarified: should we use the whole scale or follow the comments?
[a href="index.php?act=findpost&pid=287251"][{POST_SNAPBACK}][/a]


I will modify to the labels to excellent - poor.

  • Supernaut
  • [*][*][*]
64 kbps listening test 2005
Reply #110
Quote
I will modify to the labels to excellent - poor.

Perhaps you could post your choice of the labels so we can discuss and perhaps improve them, in order to make them more descriptive and as unambiguous as possible? I'm not sure "good" and "excellent" labels would help me make my results on scale with others' results...

  • Busemann
  • [*][*][*][*][*]
64 kbps listening test 2005
Reply #111
Quote
Rumour has it that Tiger will be released mid-april. Of course, Apple rumours are NEVER trustworthy.



Tiger has gone GM, the question is whether HE-AAC will be included in QT 7 after all..
  • Last Edit: 31 March, 2005, 06:21:19 PM by Busemann

  • kurtnoise
  • [*][*][*][*]
64 kbps listening test 2005
Reply #112
Sebastian...I don't know if you are already begun the preliminaries for these tests, but may I suggest to add a new candidate : Enhanced AAC+ v2 ??

Brandon Siegel has updated its CLI encoder for dbPowerAmp. It can now reach up to 64 kbps for stereo files. Check this thread directly on dBAmp forum for more infos.


Another question : have you got enough samples or can we post new ones ?

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #113
HE-AAC v.2 = AAC + SBR (v.1) + Parametric Stereo (=> v.2). According to some tests requested by Ivan ~one year ago, Parametric Stereo have a negative impact at "high" bitrate (> 40 kbps IIRC). Of course, it was noticed on Nero AAC encoder, and it doesn't necessary mean than other encoders have similar reactions. Therfore, it could be worth to test.


To be entirely honest, I don't really like the idea of testing two HE-AAC encoders (Nero & Apple) in a same multiformat test. I don't like it, but I understand the reason I think. I would rather make preliminary tests, in order to choose the best challenger for each format (as we do for wma, vorbis...). There are many possible contenders:

- Nero AAC VBR & CBR (does VBR produce stable quality at low bitrate?)
- Apple VBR & CBR (same comment)
- Coding Technologie (in Real products) AAC
- the one pointed by kurtnoise.

But to make this possible, we need people to test them first.

64 kbps listening test 2005
Reply #114
Quote
Sebastian...I don't know if you are already begun the preliminaries for these tests, but may I suggest to add a new candidate : Enhanced AAC+ v2 ??

Brandon Siegel has updated its CLI encoder for dbPowerAmp. It can now reach up to 64 kbps for stereo files. Check this thread directly on dBAmp forum for more infos.


Another question : have you got enough samples or can we post new ones ?
[a href="index.php?act=findpost&pid=287538"][{POST_SNAPBACK}][/a]


Well, with Apple and Nero, the two most popular HE-AAC encoders, I have enough HE-AAC encoders covered.

Regarding samples, I have enough now, thank you. It's a bit hard to decide which ones to use, since almost all of them are more or less killer samples.

64 kbps listening test 2005
Reply #115
Quote
To be entirely honest, I don't really like the idea of testing two HE-AAC encoders (Nero & Apple) in a same multiformat test. I don't like it, but I understand the reason I think. I would rather make preliminary tests, in order to choose the best challenger for each format (as we do for wma, vorbis...). There are many possible contenders:

- Nero AAC VBR & CBR (does VBR produce stable quality at low bitrate?)
- Apple VBR & CBR (same comment)
- Coding Technologie (in Real products) AAC
- the one pointed by kurtnoise.

But to make this possible, we need people to test them first.
[a href="index.php?act=findpost&pid=287542"][{POST_SNAPBACK}][/a]


I guess you are right, but who wants to conduce another pre-test? Look at how many people tested Vorbis.

Edit: By the way... Regarding WMA, I guess I will stick to Standard since Pro isn't really "tuned" for such bit-rates and as Roberto said, 64 kbps is used for either streaming or portable usage. None of those areas are covered well by Pro.
  • Last Edit: 01 April, 2005, 12:32:07 PM by Sebastian Mares

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #116
Quote
I guess you are right, but who wants to conduce another pre-test? Look at how many people tested Vorbis.
[a href="index.php?act=findpost&pid=287548"][{POST_SNAPBACK}][/a]


I had this in mind when I said "I don't like it, but I understand the reason I think" 
Preliminary collective tests are possible, without someone to conduce it from A to Z. Remember last 128 kbps listening test: there were many choices for vorbis (CVS, aoTuV, Quantum Knot, Modest Tuning), and the final decision was based upon personal tests. We know the result, more than positive for Vorbis

64 kbps listening test 2005
Reply #117
Quote
Quote
I guess you are right, but who wants to conduce another pre-test? Look at how many people tested Vorbis.
[a href="index.php?act=findpost&pid=287548"][{POST_SNAPBACK}][/a]


I had this in mind when I said "I don't like it, but I understand the reason I think" 
Preliminary collective tests are possible, without someone to conduce it from A to Z. Remember last 128 kbps listening test: there were many choices for vorbis (CVS, aoTuV, Quantum Knot, Modest Tuning), and the final decision was based upon personal tests. We know the result, more than positive for Vorbis
[a href="index.php?act=findpost&pid=287549"][{POST_SNAPBACK}][/a]


Yes, but who has the time to spend on testing? I don't think I can justify a winner based on five or six test results.

Edit: BTW, everyone is free to conduce pre-tests.
  • Last Edit: 01 April, 2005, 12:37:52 PM by Sebastian Mares

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #118
Quote
I don't think I can justify a winner based on five or six test results.
[a href="index.php?act=findpost&pid=287550"][{POST_SNAPBACK}][/a]


I can't find the exact thread, but if I remember correctly, there were no more persons to test various fork of Vorbis in order to select the good one last spring.

Quote
Yes, but who has the time to spend on testing?

Time is not the problem. Motivation is probably a greater one. I don't think that many people will be interested to test HE-AAC implementation that are not used otherwise.

Nevertheless, removing one redundant challenger could make the test easier (each additionnal codec makes evaluation and the establishment of the hierarchy harder). There are currently 7 contenders (including anchor): that's very much, especially for untrained people.

64 kbps listening test 2005
Reply #119
Quote
Nevertheless, removing one redundant challenger could make the test easier (each additionnal codec makes evaluation and the establishment of the hierarchy harder). There are currently 7 contenders (including anchor): that's very much, especially for untrained people.
[a href="index.php?act=findpost&pid=287559"][{POST_SNAPBACK}][/a]


Well, the most I can do is dump ATRAC3+ and if enough tests are available, one of the HE-AAC encoders.
The reason why I wanted both HE-AAC encoders to be featured is that a lot of work was done to improve the Nero encoder and it is also interesting to see how Apple performs against the big competitor (so there is a direct comparison).

64 kbps listening test 2005
Reply #120
Quote
I can't find the exact thread, but if I remember correctly, there were no more persons to test various fork of Vorbis in order to select the good one last spring.
[{POST_SNAPBACK}][/a]


[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=20389]This?[/url]

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64 kbps listening test 2005
Reply #121
Yes, this one. I've found the exact post I had in mind. Roberto had only 5 set of results (+ 2 partial results) to make a decision. Also note that aoTuV beta 1 was pre-tested, and that the non-tested aoTuV beta2 was included in the final collective test... and won (tied with mpc). Taking some risks is not necessary a bad thing
  • Last Edit: 01 April, 2005, 01:24:30 PM by guruboolez

  • rudefyet
  • [*][*][*]
64 kbps listening test 2005
Reply #122
Quote
- Nero AAC VBR & CBR (does VBR produce stable quality at low bitrate?)


From personal tests the "fast" mode on Nero causes some serious problems/distortions around 96kbps or lower in LC-AAC and somewhere below 64kbps it causes trouble on HE-AAC (32kbps HE-AAC in fast mode sounded horrific! High mode sounded much better)

high mode with the "streaming" profile gives good quality 64kbps HE-AAC, fast mode sounds similar at 64kbps, but the bitrates only average around ~50-55kbps, i haven't done any ABxing to determine any quality differences though
  • Last Edit: 12 April, 2005, 09:50:49 PM by rudefyet

  • negritot
  • [*][*][*]
64 kbps listening test 2005
Reply #123
So Quicktime 7 will be widely available in a little over two weeks. Is that enough time to narrow down the contenders? And can anyone verify that Quicktime 7 includes an HE-AAC encoder? AAC isn't even mentioned on the preview pages for Quicktime 7.

64 kbps listening test 2005
Reply #124
Quote
So Quicktime 7 will be widely available in a little over two weeks. Is that enough time to narrow down the contenders? And can anyone verify that Quicktime 7 includes an HE-AAC encoder? AAC isn't even mentioned on the preview pages for Quicktime 7.
[a href="index.php?act=findpost&pid=290173"][{POST_SNAPBACK}][/a]


Well, I am still waiting for some Vorbis tests. Now that AoTuV PB 4 is out, somebody might want to test that against Xiph 1.1 and Archer.
Regarding HE-AAC tests, I doubt that anyone would like to test on their own. Also, I would rather get the test started pretty soon after Apple releases their HE-AAC encoder, so there won't be much room (time) for pre-tests. Therefore, I guess I will stick to my first decision featuring both codecs in the main test.