It’s time to discuss the conditions of new public test.
At least for now there won't be poll vote about possible codecs to test because it will be better to speak personally.
The polls can't tell us if the person had participated or want to participate in such tests.
Previous test at 64 kbps wasn’t difficult but neither very easy for everyone. So 96 kbps is a reasonably good bitrate for new test. 128 kbps have been already tested in previous tests.
In my opinion now we have 2 possible tests:
LC-AAC test ~ 96 kbps
Multiformat ~ 96 kbps
One will be performed during July –August of this year and another in next year.
I will strongly suggest to perform first LC-AAC test and only then go for Multiformat.
Because later it can turn into a lot of discussion of not right choice of AAC encoder for Multiformat test.
Second reason is that a lot of work has been already made on future AAC test
Samples and encoders have been alearedy defined in previous discussion.
LC-AAC encoders to test:
1. Nero
2. Apple true VBR
3. Apple iTunes constrained VBR
4. Winner between Winamp's Coding Tehcnologies and Divx (Pre-test)
See here http://www.hydrogenaudio.org/forums/index....c=77272&hl= (http://www.hydrogenaudio.org/forums/index.php?showtopic=77272&hl=)
Only 4 codecs should be inlcuded (4 + low anchor). Generally too much codecs will lead to loss of listener's concentration.
Some members of HA (Garf, Chris, Alex B, /mnt ) are welcomed to be co-organizers or simply observers of this test to control e-mail communication with listeners and other conditions (samples, bitrate verifications, packages etc.)
The public test will be started during the end of July, 2011. We have more than one month to discuss all conditions.
Please, share your thoughts here.
P.S. Also it will be nice if someone who knows HTML will help to edit some web pages.
P.S.2. Opinions of the listeners from the last public test will be especially valuable for me.
Kinda Bump!
if it will be AAC public test then probably samples are pretty defined. http://www.hydrogenaudio.org/forums/index....st&p=695576 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=695576)
The quality of the codecs doesn't change from that point.
Unless somebody has different point of view.
I have always wanted to see a test of constrained vs true VBR on non "killer" samples.
I would love to participate. Now I have a pair of Sennheiser HD 600 and a Little Dot Mk IV which ought to help with critical listening.
I'd love to see how faac (http://sourceforge.net/projects/faac) stands up to current encoders. The development seems to be idle right now, but I'd like to know the current status in comparison to the alternatives, since I see it being recommended once in a while in #ffmpeg.
I would love to participate. Now I have a pair of Sennheiser HD 600 and a Little Dot Mk IV which ought to help with critical listening.
Nice.
I'd love to see how faac (http://sourceforge.net/projects/faac) stands up to current encoders. The development seems to be idle right now, but I'd like to know the current status in comparison to the alternatives, since I see it being recommended once in a while in #ffmpeg.
FAAC 1.23.5 has been included in one of the public tests some time ago. http://listeningtests.t35.com/html/AAC_at_...est_results.htm (http://listeningtests.t35.com/html/AAC_at_128kbps_v2_public_listening_test_results.htm)
The current version is 1.28
I'm not aware if there are any substantial improvements since then.
The good news that probably we will see new Fraunhofer (FhG) AAC encoder in this test.
In this case we should redefine the samples because new codec will be included.
People, prepare to suggest your samples if it will happen. It won't be me who will choose samples.
The good news that probably we will see new Fraunhofer (FhG) AAC encoder in this test.
Sounds interesting, i've never seen any tests from a Fhg AAC encoder.
From old test
Sorenson Squeeze 3.5 (FhG Pro)
http://listeningtests.t35.com/html/AAC_at_...est_results.htm (http://listeningtests.t35.com/html/AAC_at_128kbps_public_listening_test_results.htm)
From old test
Sorenson Squeeze 3.5 (FhG Pro)
http://listeningtests.t35.com/html/AAC_at_...est_results.htm (http://listeningtests.t35.com/html/AAC_at_128kbps_public_listening_test_results.htm)
Opps, i forgot that the Sorenson encoder was a Fhg encoder.
Though it must have been a really old Fraunhofer encoder. That test is from June 2003*. In 2005, Fraunhofer started rewriting its AAC encoder more or less from scratch, so a lot has changed.
The good news that probably we will see new Fraunhofer (FhG) AAC encoder in this test.
In this case we should redefine the samples because new codec will be included.
Why? Back in early 2010 I spent weeks digging through old HA test material as well as new material suggested by /mnt and others to
- find samples which highlight different types of artifacts and aspects of encoding quality (see the link you mentioned above),
- amplified or attenuated them so they won't clip upon decoding (a thing which had not been done in the 2011 64-kb test!),
- limit them to 15 seconds to minimize listener fatigue (also not done in the 2011 test).
Bashing that entire test set would be a waste of resources. That being said, we could think about adding some samples from the 2011 test. Say, 5 samples to make it a total of 20. Pure spoken or sung vocals, for example, have not been considered in the 2010 test set. And we could think about replacing non-music/non-vocal samples such as applause by more musical/vocal items.
Chris
* at least that's what the file dates in the RARed comments say
Chris,
You (Fraunhofer's developer) suggest to include your set of samples and include your Fraunhofer AAC encoder into test.
Is it what you want?
Let's put time limits to decide what codecs to test.
Starting from 1st of July no new competitor codec will be accepted to be included. Without exception.
Then we will discuss the choice of samples and settings.
Edit: Grammar
Temporally removed.
Couldn't iTunes "constrained" versus "true" VBR be done as a sub-test? Obviously I'm not the most unbiased commentator for a listening test, but it seems excessive. Unless it's something that the HA community truly doesn't know which is better.
Couldn't iTunes "constrained" versus "true" VBR be done as a sub-test? Obviously I'm not the most unbiased commentator for a listening test, but it seems excessive. Unless it's something that the HA community truly doesn't know which is better.
Good point.
Although many people want to see comparison between these two VBR modes (as it's indicated in previous discussion past year) it will be difficult to compare them because true vbr doesn't produce the same bitrate as constrained VBR.
true VBR (--tvbr 45) - ~92 kbps
constrained VBR ~100 kbps
The list of candidate codecs for ~100 kbps test:
1. Nero
2. At least one Apple encoder (CVBR or TVBR) or two of them (?)
3. Fraunhofer
4. Winamp's Coding Technologies and Divx (Pre-Test)
Fraunhofer encoder will be publicly available on 30th of June.
All codecs except true VBR can produce 100 kbps for this test.
I expect suggestions from community.
First, it's unacceptable to have the set of the samples provided by the developer of one of included competitors.
Nobody won't tolerate it. If these samples would be included later everybody will accuse me to be not impartial.
Second, first 50% of samples for previous test were taken from Sebastian Mares's 64 kbps and very few from Gabriel 48 kbps test and second 50% were new. First and second halfs have shown the same results.
You can verify for yourself.
I'm standing on right place.
P.S.The choice of samples, codecs, conditions and bitrate verification are public.
I would very much like to see a multiformat 96kbps test that includes both Nero AND Apple (Quicktime) LC-AAC encoders.
Ok, your vote counts.
Let's see if there will be more people who are interested in multiformat.
As I understand it, transparency for the latest batch of codecs is believed to be somewhere between 128 kbit and 64 kbit, thus the proposed testing at 96 kbit.
As we try to resolve this further, we should also try to deal with the fact that outside scientific and developer circles, transparency at 128 kbit is not well appreciated. Is there some testing we can do that will bring a greater appreciation?
Many people base their disdain for lossy coding on the performance of early encoders. Can we, to show how far the art has progressed, compare some crusty but widely-used encoder at high bit rate to newer ones at lower bit rates?
Many people base their disdain for lossy coding on the performance of early encoders. Can we, to show how far the art has progressed, compare some crusty but widely-used encoder at high bit rate to newer ones at lower bit rates?
Good suggestion. I believe that it is mp3 that is still the main (and maybe the only) choice outside of scientific and developer circles.
And at the same bit rate, there's quite a range of MP3 encode quality depending on encoder vintage and settings. Most users assume they get the same quality if they use the same file format with the same bitrate.
First, it's unacceptable to have the set of the samples provided by the developer of one of included competitors.
Nobody won't tolerate it. If these samples would be included later everybody will accuse me to be not impartial.
"My" 2010 testset (http://www.hydrogenaudio.org/forums/index.php?showtopic=77584&st=50&p=695576&#entry695576). . . "Igor's" 2011 testset (http://listening-tests.hydrogenaudio.org/igorc/results.html) (and/or who suggested the item for "my" testset)
AngelsFallFirst. . . . not present, but very similar to Sample01
Robots . . . . . . . . Sample05 (the 1978 un-remastered version, 2009 remaster suggested by /mnt (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681472))
Kalifornia . . . . . . Sample10 (longer than Kalifornia, a.k.a. Fatboy, suggested by IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517))
Linchpin . . . . . . . Sample17 (longer version, suggested by /mnt and IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681472)). FallOfLife not present
Waiting. . . . . . . . Sample18 (suggested by IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517))
Applause, Harlem . . . not present, applauses (http://ff123.net/samples.html) from at least 2004 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=20504&view=findpost&p=201604), but I suggested to remove it from my set (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=89208&view=findpost&p=760274). (Applause was supported by rpp3po (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=688878))
Can't Wait . . . . . . not present, item from 2004 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&st=0&p=195478&#entry195478)
FallOfLife . . . . . . not present, item from 2004 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=19882&view=findpost&p=195618)
Rumba. . . . . . . . . not present, item from 2004 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=19882&view=findpost&p=201619)
SQAM Selection . . . . not present, items from 1988, publicly released 2008 (http://tech.ebu.ch/publications/sqamcd)
Memories . . . . . . . not present (suggested by naturfreak (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=692203))
Hancock. . . . . . . . not present (suggested by IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517))
Creuza de Mä . . . . . not present (suggested by IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517))
Trumpet. . . . . . . . not present (suggested by halb27 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681449))
BerlinDrug . . . . . . not present (suggested by Bryanhoop (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=682234))
Girl . . . . . . . . . not present (suggested by IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=683378))
Ecstasy. . . . . . . . not present (suggested by IgorC (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=698545))
P.S. The choice of samples, codecs, conditions and bitrate verification are public.
Sorry if I missed it, but where was the public selection and discussion of the test items for your 2011 test at 64 kbps?
This will be my last post in this thread or any other related to this public test.
Chris
The order of events:
1. The selection of the encoders.
2. Only then we should choose samples (if they are new competitors).
It doesn't matter who submit the samples.
The order of events:
1. The selection of the encoders.
2. Only then we should choose samples (if they are new competitors).
It doesn't matter who submit the samples.
So, is the Fhg encoder in or out?
So, is the Fhg encoder in or out?
It's in. People decide it.
IgorC, and anyone else, are free to point out that a certain cause of action might disproportionally benefit a certain encoder. But you can expect every developer to make recommendations that help rather than hurts their cause. This is natural. If the process is open, the community here will identify such situations and call to correct them. This also ensures that in the case you, the organizer of the test does something like that, it is identified and you can correct it so the test stays neutral.
I wasn't entirely happy with how the samples in the past test were selected, but told you I thought it was reasonable enough that people would accept it. You're free to observe how the post-test discussion regarding that subject went. There is some irony you're now upset at another persons recommendation of samples.
If you attack a codec developer, the result is entirely predictable: they will walk away and point to the attacks as evidence the test was biased against them. (And note this will happen regardless if the end result ends up being biased or not) This takes away from the people who help with the effort to organize the test, and of the people who spend time on taking it.
If I'm a person interested in taking the test, and I come to this thread and see the organizer attacking one of the participants, what do you think I'm going to do?
I'm going to leave this thread open because I'm interested in what people want to see tested. But I'm also going to moderate it if the discussion is not conducted in a civilized manner.
If the process is open,
Indeed, yes, it's open
they will walk away and point to the attacks as evidence the test was biased against them.
The choice of conditions (samples, bitrate, settings) will be made by all members (except some obvious restrictions as it has happened) . So there is no chance to get it biased way.
Yes, I perfectly admit that I was harsh. I won't repeat such thing again. But this test stands for human values not commercial interests.
Guys, this test will be for You and made by You. You will choose all samples, You will choose all codecs, You will choose all settings, You will control all conditions and You will test it.
I'm actually more interested on multiformat.
In case of multiformat there are next possibilties:
Vorbis AoTuV, AAC (which one ?), CELT/Opus (?), MP3 (?)
Let's decide if test will be multiformat or AAC. Deadline was set until 1st of July
Right now Kennedyb4, Kohlrabi(?), /mnt are interested in AAC test.
Kwanbis and Zarggg - multiformat
Who else?
The list of codecs for ~100 kbps AAC test:
Candidates:
1. Nero
2. At least one Apple encoder (CVBR or TVBR) or two of them (?)
3. Fraunhofer
4. Winamp's Coding Technologies and Divx (Pre-Test)
As I understand it, transparency for the latest batch of codecs is believed to be somewhere between 128 kbit and 64 kbit, thus the proposed testing at 96 kbit.
As we try to resolve this further, we should also try to deal with the fact that outside scientific and developer circles, transparency at 128 kbit is not well appreciated. Is there some testing we can do that will bring a greater appreciation?
Many people base their disdain for lossy coding on the performance of early encoders. Can we, to show how far the art has progressed, compare some crusty but widely-used encoder at high bit rate to newer ones at lower bit rates?
Last Sebastian's test had the old l3enc MP3 encoder. It showed how bad can be this old mp3 encoder in comparison with the modern MP3 encoder.
http://listening-tests.hydrogenaudio.org/sebastian/ (http://listening-tests.hydrogenaudio.org/sebastian/)
I will strongly suggest to perform first LC-AAC test and only then go for Multiformat.
Because later it can turn into a lot of discussion of not right choice of AAC encoder for Multiformat test.
Although I am solely interested in the multiformat test, I have to agree with you here, IgorC. I'd say each audioformat in the multiformat test needs to be represented by the best encoder around.
Another aspect I'm really interested in; besides all ~96kbps samples, including some 128kbps samples encoded with an old mp3 encoder. I think this even more interests me than a battle between ~96kbps samples encoded with all sorts of state-of-the-art encoders. How cool would it be seeing a ~96kbps sample outperforming an "old" 128kbps sample!
Hi. Sorry for my bad English.
In my opinion we should perform 96 kbps multiformat testing with next encoders:
Vorbis aoTuV 6.03
Nero AAC 1.5.4.0 (or maybe QT AAC, unfortunately I'm not sure too)
CELT, of course
LAME 3.98.4 (but something tells me that we should try FhG for this bitrate. at least for me it sounds better at 128 kbps)
Can we include more then one encoder per format into the test? Otherwise we must first of all determine the best encoder @96kbps for mp3 and AAC. I see no other ways.
And how about WMA (or even WMA 10 Pro)?
How cool would it be seeing a ~96kbps sample outperforming an "old" 128kbps sample!
I cannot see a practical side in this...
BTW, Nero also has several modes: vbr, cbr, abr, abr 2pass...
Let's decide if test will be multiformat or AAC. Deadline was set until 1st of July
Right now Kennedyb4, Kohlrabi(?), /mnt are interested in AAC test.
Kwanbis and Zarggg - multiformat
Who else?
I'm also only interested in a multiformat test.
Can we include more then one encoder per format into the test? Otherwise we must first of all determine the best encoder @96kbps for mp3 and AAC. I see no other ways.
Yes, it's possbile to include 2 encoders of the same format but then we should exclude one of the formats. The max. number of codecs should be 4. The previous tests indicates that it's affordable number of codecs to test.
And how about WMA (or even WMA 10 Pro)
Statistics indicates that very very few persons who use it. http://www.hydrogenaudio.org/forums/index....showtopic=86830 (http://www.hydrogenaudio.org/forums/index.php?showtopic=86830)
I'm also only interested in a multiformat test.
ok, but I will ask people to specify also encoders for multiformat and/or AAC candidate tests.
It should be highlighted that if multiformat will be conducted first then AAC test will be in next year.
The only thing that changes is chronological order.
Current update:
Multiformat - Zarggg, Kwanbis, dsimcha - (3)
AAC - Kennedyb4, Kohlrabi, /mnt - (3)
Multiformat / but realize that it will be difficult to choose best encoder per format - Corone, Steve Forte Rio - (2)
Who else?
I do not like WMA at all, but i find it more interesting to test it than lets say CELT. I would do
MP3 LAME
AAC iTunes
Vorbis AoTuv
WMA
I am for an AAC test.
I am for an AAC test.
Me too. There a lot of news for AAC world. Last Nero, iTunes and new FhG. Damn, it would be great.
I've received preview version of FhG. it looks interesting.
What about Nero encoder? 1.5.4 is ok, right?
I do not like WMA at all, but i find it more interesting to test it than lets say CELT. I would do
MP3 LAME
AAC iTunes
Vorbis AoTuv
WMA
I guess it's WMA PRO 10. Right?
AAC test would be great.
If so, I'm for an AAC test too.
But what dou you think about competition between:
Nero AAC, QT AAC, LAME MP3, FhG MP3?
Thus we can find out the winner for mp3 and aac using the results of only one test. Or someone insist on iclusion other encoders to the AAC test?
AAC test would be great.
If so, I'm for an AAC test too.
Great.
But what dou you think about competition between:
Nero AAC, QT AAC, LAME MP3, FhG MP3?
Thus we can find out the winner for mp3 and aac using the results of only one test. Or someone insist on iclusion other encoders to the AAC test?
Here I really don't know what to say. You can suggest this set of codecs and we will see if some other members will be interested to test these encoders.
But I think LAME and FhG encoder have been already compared between themselves in one of Sebastian's public test. Both were tied.
So what encoder we should choose for our test - LAME or FhG? Сonsidering the results for 128 kbps test, I suppose it is LAME 3.98
And I'm still don't see the final solution about the list of formats (for future multiformat test) - will it be aac/celt/mp3/vorbis or aac/celt/vorbis/wma or some else combination?
I vote for doing a AAC test only first, even though I currently don't use it.
Later on, multiformat would be fine.
Cheers
johnb
So what encoder we should choose for our test - LAME or FhG?
And I'm still don't see the final solution about the list of formats (for future multiformat test) - will it be aac/celt/mp3/vorbis or aac/celt/vorbis/wma or some else combination?
Well, we are in the middle of suggestion period for set of codecs. People make suggestions and the next weekend all of us will analyse suggestions and will make final conclusion. Let's wait for suggestions of other members.
Ok, then my final suggestion:
1) AAC Test:
Nero AAC
QT AAC
Fraunhofer
CT AAC (Winamp)
2) Multiformat test:
AAC (winner)
CELT
Vorbis aoTuV
WMA Pro
How can a multi format test be considered complete without testing MP3 alongside the others?
I'm also in favour of an (LC-)AAC test. As on the AAC side, there are probably only 3 or 4 contenders for the crown, amongst which Nero and Apple, therefore I agree that it would be interesting to compare them to some vintage MP3 codec at 128k CBR, such as Fraunhofer's L3enc or MP3enc, as a 4th or 5th contender in the test.
I also still volunteer to host the test samples on my web server, as I did a couple of years ago for the 128k MP3 test.
I think an LC-AAC test would nice and then we can use the winner in a mult-format test
For the multi-format test I would prefer constrained VBR or CBR at 96Kb/s for streaming purposes.
- CELT/OPUS
- Vorbis aoTuV
- LAME MP3
- Winner of the LC-AAC test
I guess it's WMA PRO 10. Right?
Correct.
How can a multi format test be considered complete without testing MP3 alongside the others?
Exactly my thoughts.
For the multi-format test I would prefer constrained VBR or CBR at 96Kb/s for streaming purposes.
Do people actually care bout streaming quality that much? I tend to think that people that care about quality would not be listening to 96kbps music.
I will include suggestion of developers as well. It won't be right just don't count them. Later we will see what should be done in this case.
Also if members have voted for Apple encoder I submit them to iTunes (CVBR) because until now TVBR produces very low bitrate for the test (~92 kbps).
Current votes:
Multiformat - Zarggg, Kwanbis, dsimcha, (3)
AAC - Kennedyb4, , /mnt, muaddib, Steve Forte Rio, IgorC, Pri3st, johnb, Polar, LithosZA - (8 +1 developer)
Multiformat / but realizes that it will be difficult to choose best encoder per format - Corone (1)
Not sure what Kohlrabi wants.
Codecs:
MP3 LAME – Kwanbis, Nick.C, LithosZA, IgorC (4)
Vorbis AoTuV – Kwanbis, LithosZA, IgorC, Steve Forte Rio (4)
CELT – Steve Forte Rio, LithosZA, IgorC, (3)
WMA/WMA Pro – Kwanbis, Steve Forte Rio, (2)
iTunes AAC (constrained VBR) - Kennedyb4, Zarggg, Kwanbis, Steve Forte Rio, IgorC, Polar (6)
Nero – Zarggg, Steve Forte Rio, IgorC, Polar (4)
FhG AAC - /mnt, Steve Forte Rio, IgorC (3)
Winamp’s CT AAC – Steve Forte Rio, IgorC (2)
QuickTime AAC (true VBR) – Kennedyb4, benski (1 + 1 developer against)
FAAC – Kohlrabi (1)
Old encoder with higher bitrate – Notat, Corone, Polar, (3)
To include only one Apple encoder in AAC test (CVBR or TVBR) – IgorC, Benski (2) (1 + 1 developer)
Who else?
Also if members have voted for Apple encoder I submit them to iTunes (CVBR) because until now TVBR produces very low bitrate for the test (~92 kbps).
Hm, didn't know it. But in my opinion constraining of the bitrate isn't a good idea. Please, move my vote from CVBR to TVBR.
Me multiformat including at least:
MP3 LAME
Nero
QuickTime AAC (true VBR)
and optionally, preferring Vorbis:
Vorbis AoTuV
iTunes AAC (constrained VBR)
QuickTime AAC (true VBR) or iTunes AAC (constrained VBR), prefer TVBR
Nero
FhG AAC
Lame mp3
I am for an AAC test.
What about Nero encoder? 1.5.4 is ok, right?
I am not working for Nero anymore.
For the AAC test i would chose:
Nero AAC
QuickTime AAC / True VBR
FhG AAC
DivX labs AAC encoder or Winamp's AAC encoder (pre-test)
FAAC
For a multi codec test, i would chose:
Winner of the AAC test.
CELT
Vorbis AuTov
LAME 3.99 beta with --vbr-new
Musepack
for multicodec:
lame
aoTuV
AAC
WMA
I am not working for Nero anymore.
oh. I wish You the best in your new activities.
Please, check your votes and tell me if there is any error.
Now the numbers of suggestions are acceptable.
Current votes:
Multiformat - Zarggg, Kwanbis, dsimcha, Alexxander, romor (5)
AAC - Kennedyb4, /mnt, muaddib, Steve Forte Rio, IgorC, Pri3st, johnb, Polar, LithosZA, richard123 - (10)
Multiformat / but realizes that it will be difficult to choose best encoder per format - Corone (1)
Not sure what Kohlrabi wants.
Codecs:
MP3 LAME – Kwanbis, Nick.C, LithosZA, IgorC, Alexxander, richard123, /mnt, romor (8)
Vorbis AoTuV – Kwanbis, LithosZA, IgorC, Steve Forte Rio, Alexxander (opt.), /mnt, romor (6 +1 optionally)
CELT – Steve Forte Rio, LithosZA, IgorC, /mnt (4)
WMA/WMA Pro – Kwanbis, Steve Forte Rio, romor (3)
Musepack (MPC) - /mnt (1)
Nero – Zarggg, Steve Forte Rio, IgorC, Polar, Alexxander, richard123, /mnt (7)
iTunes AAC (constrained VBR) - Kennedyb4, Zarggg, Kwanbis, IgorC, Polar, Alexxander (optinally) (5 + 1 optionally )
QuickTime AAC (true VBR) – Kennedyb4, Steve Forte Rio, Alexxander, richard123, /mnt (5)
FhG AAC - /mnt, Steve Forte Rio, IgorC, richard123 (4)
Winamp’s CT AAC – Steve Forte Rio, IgorC, /mnt (3)
FAAC – Kohlrabi, /mnt (1)
Divx AAC - /mnt (1)
Old encoder with higher bitrate – Notat, Corone, Polar, (-)lvqcl (3 + 1 against )
To include only one Apple encoder in AAC test (CVBR or TVBR) – Benski (1 developer)
Who else?
Edit: there was one error in vote list.
IMO it is a waste of time to test FAAC and DivX, and it would be better to have only one iTunes.
Maybe a pretest for iTunes or a suggestion from an Apple developer?
So for me, it is: Nero, FhG, CT and iTunes (either CVBT or TVBR).
Vote for AAC test.
And I also have a question: what software uses/will use new FhG encoder?
@muaddib and lvqcl
I will update the list of votes with your choices later.
IMO it is a waste of time to test FAAC and DivX, and it would be better to have only one iTunes.
Maybe a pretest for iTunes or a suggestion from an Apple developer?
People show a little interest in FAAC and Divx. It will be safe to just exclude them.
Now it makes me think that pre-test is a bad idea. If one codec will pass the pre-test then later listeners will be more familiarized with its specific artifacts. So test will be biased against this particular codec. This has happened in one of Ivan's public test where high anchor (MP3) was tested twice. It had no any impact as it was only high anchor. http://www.hydrogenaudio.org/forums/index....c=41191&hl= (http://www.hydrogenaudio.org/forums/index.php?showtopic=41191&hl=)
BTW I've seen your work on perception of sound . Interesting.
Vote for AAC test.
And I also have a question: what software uses/will use new FhG encoder?
I can't comment right now as it's not my competence. Unless developers will do their own statements.
Though as I already have mentioned it will be available for all users on 30th of June.
I will ask people to not discuss the quality of any of the codecs right now to avoid any kind of personal prejudices during the ABC/HR sessions of this public test.
P.S. There is a lot of interest to see CVBR (popular because of itunes) against TVBR (possible superiority).
People that have interest can also do a listening test comparing iTunes CVBR and TVBR after the first AAC listening test.
Then also another AAC encoder (or the same high anchor if there will be one) could be included in that CVBR vs TVBR test to get a scaling comparable to the first AAC test.
@IgorC: Thanks to all your nice words
@muaddib
It's ok for me to include only one (CVBR or TVBR) mode. Or both of them.
Let's see the suggestions of other members.
Anyway we will return to this question next week during bitrate verification. TVBR and CVBR produce different bitrates.
Current votes
http://listening-test.blogspot.com/2011/06...reparation.html (http://listening-test.blogspot.com/2011/06/discussion-on-preparation.html)
If member hasn't specify whether he suggests CVBR or TVBR then it's half vote for each codec (0.5 vote for CVBR, 0.5 vote for TVBR)
People, what do you think about CVBR and TVBR? To include both of them or just one? If only one then which (CVBR or TVBR)?
But we can test only 4 codecs.
Suggestions, please.
Basic plan:
Decide what to test (AAC or Multiformat) and have an idea what codecs - until July 1
Bitrate verification - 7-10 days - until July 7-10
Sample selection - 7-10 days - until July 15-20
Checking all conditions, preparations, dummy packages etc. Developers can also take a look if everything is alright. - until July 23-24
Start of test - last week of July.
People, what do you think about CVBR and TVBR? To include both of them or just one? If only one then which (CVBR or TVBR)?
Only one, of course. I suppose, it should be TVBR
For a multi codec test, i would chose:
Winner of the AAC test.
CELT
Vorbis AuTov
LAME 3.99 beta with --vbr-new
Musepack
At the target bitrate of 96 kbps Musepack would be clearly inferior, I believe. It is just not supposed to be transparent there.
-1 vote
I don't think it's a waste of time to test FAAC, by far most AAC content is encoded with it, especially in the 96kbps range. Testing FAAC can provide clear and up to date argumentation for why many internet services should or shouldn't keep using a free solution as opposed to a commercial solution.
Of course my vote doesn't count, but just trying to provide some arguments for other people to make a decision
Just been forced to use ffmpeg's 0.8 aac encoder (libav52.xxx) while i was transcoding some videos for the iPod, it's a perfect canidate for a low anchor AAC encoder.
For a multi codec test, i would chose:
Winner of the AAC test.
CELT
Vorbis AuTov
LAME 3.99 beta with --vbr-new
Musepack
At the target bitrate of 96 kbps Musepack would be clearly inferior, I believe. It is just not supposed to be transparent there.
-1 vote
So is MP3.
Just been forced to use ffmpeg's 0.8 aac encoder (libav52.xxx) while i was transcoding some videos for the iPod,
MeGui has a lot of profiles (iPod as well) and uses high quality encoders x264 and Nero AAC.
I don't think it's a waste of time to test FAAC, by far most AAC content is encoded with it, especially in the 96kbps range. Testing FAAC can provide clear and up to date argumentation for why many internet services should or shouldn't keep using a free solution as opposed to a commercial solution.
Of course my vote doesn't count, but just trying to provide some arguments for other people to make a decision
it's a perfect canidate for a low anchor AAC encoder.
FAAC could be low anchor maybe at 65-70 kbps (?). Low anchor should have clearly inferior quality. It's important at the time to analyze the results.
Current votes:
http://listening-test.blogspot.com/2011/06...reparation.html (http://listening-test.blogspot.com/2011/06/discussion-on-preparation.html)
Just been forced to use ffmpeg's 0.8 aac encoder (libav52.xxx) while i was transcoding some videos for the iPod,
MeGui has a lot of profiles (iPod as well) and uses high quality encoders x264 and Nero AAC.
I have tried MeGui, but i prefer to just using CLI encoders for video, such as ffmpeg and menconder. I have been working on a FFmpeg frontend in C#, which is designed to have a basic load file and select preset interface, however the Windows builds of ffmpeg on the internet are usually out of date or lacking libs such as libfaac or libx264.
ffmpeg has its own AAC encoder which is not FAAC. Last time I checked these two encoders have comparable quality, don't they?
ffmpeg has its own AAC encoder which is not FAAC. Last time I checked these two encoders have comparable quality, don't they?
ffmpeg's AAC encoder sounds far worse then FAAC to me:
foo_abx 1.3.4 report
foobar2000 v1.1.7
2011/06/29 00:41:42
File A: C:\Temp\Derezzed_ffmpegAAC.m4a
File B: C:\Temp\Derezzed_FAAC.m4a
00:41:42 : Test started.
00:41:56 : 01/01 50.0%
00:42:03 : 02/02 25.0%
00:42:14 : 03/03 12.5%
00:42:22 : 04/04 6.3%
00:42:31 : 05/05 3.1%
00:42:40 : 06/06 1.6%
00:42:46 : 07/07 0.8%
00:42:50 : 08/08 0.4%
00:42:59 : 09/09 0.2%
00:43:07 : 10/10 0.1%
00:43:13 : 11/11 0.0%
00:43:19 : 12/12 0.0%
00:43:26 : 13/13 0.0%
00:43:33 : 14/14 0.0%
00:43:39 : 15/15 0.0%
00:43:47 : 16/16 0.0%
00:43:54 : 17/17 0.0%
00:43:56 : Test finished.
----------
Total: 17/17 (0.0%)
Both tracks are encoded at 128kbps ABR.
The ffmpeg encode has very harsh disortion and warbling. File B sounds better, but the warbling and the pre-echo at the start of the track makes it far from transparent.
Thank you, /mnt
What build do you use and settings?
Low anchor probably should be AAC with low quality and bitrate 96 or less.
GXLAME is a fork of LAME encoder. It's tuned for low bitrates.
http://www.hydrogenaudio.org/forums/index....80510&st=25 (http://www.hydrogenaudio.org/forums/index.php?showtopic=80510&st=25)
http://www.hydrogenaudio.org/forums/index....mp;#entry754931 (http://www.hydrogenaudio.org/forums/index.php?showtopic=80509&st=0&p=754931&#entry754931)
Multiformat gets my vote, I would be intrested to see aac, vorbis, lame and celt. But then again, why not get the best of both worlds and test a few aac encoders at the same time?
I'm with Kef here, I vote for a multiformat test with multiple AAC encoders. Something like LAME, AoTuV, CELT, iTunes AAC, Nero AAC. (was there a listening test with Musepack SV8 in it, yet?)
@Kef and Kohlrabi
Thank you for you votes. I will update the list of votes today.
Why not get the best of both worlds and test a few aac encoders at the same time?
This is what we are trying to figure out. The votes speak for themselves. A lot of people want AAC test to figure out more/most optimal AAC encoder(s). At the same time there are enough people who want to see multiformat test.
What is your suggestion for selection of AAC encoders and what are more important factors for such decisions (popularity, quality and availability of encoder)?
Something like LAME, AoTuV, CELT, iTunes AAC, Nero AAC. (was there a listening test with Musepack SV8 in it, yet?)
5-6 codecs are too much for one test. Everything will be tied.
We will hardly get statistical difference with such high amount of codecs.
4 is still affordable.
@/mnt
If ffmpeg's AAC encoder is pretty bad then 128 kbps will be good.
What do you think?
Then it will fulfill this request. Two birds with one stone.
Old/low quality encoder with higher bitrate – Notat, Corone, Polar, (-)lvqcl (3 + 1 against )
Thank you, /mnt
What build do you use and settings?
Low anchor probably should be AAC with low quality and bitrate 96 or less.
I was a using Windows build from git at 26/06/2009, which lacked libfaac support.
The settings was:
ffmpeg -i file -acodec aac -ab 128k -strict experimental file.m4a
@/mnt
If ffmpeg's AAC encoder is pretty bad then 128 kbps will be good.
What do you think?
Then it will fulfill this request. Two birds with one stone.
ffmpeg's AAC encoder seems to produce far worse results then FAAC, it's AAC's answer to Blade.
I have done a new test with the offical stable release of ffmpeg 0.8 on Ubuntu 10.04 (my own compile with libx264, libxv264, libfaac and libmp3lame enabled).
ffmpeg AAC settings:
ffmpeg -i file -acodec aac -ab 128k -strict experimental file.m4a
ffmpeg FAAC settings:
ffmpeg -i file -acodec libfaac -ab 128k file-faac.m4a
foo_abx 1.3.4 report
foobar2000 v1.1.7
2011/06/29 15:42:18
File A: C:\Temp\Linchpin (ffmpeg AAC).m4a
File B: C:\Temp\Linchpin (ffmpeg FAAC).m4a
15:42:18 : Test started.
15:42:30 : 01/01 50.0%
15:42:35 : 02/02 25.0%
15:42:41 : 03/03 12.5%
15:42:48 : 04/04 6.3%
15:42:53 : 05/05 3.1%
15:42:57 : 06/06 1.6%
15:43:05 : 07/07 0.8%
15:43:08 : 08/08 0.4%
15:43:13 : 09/09 0.2%
15:43:16 : 10/10 0.1%
15:43:21 : 11/11 0.0%
15:43:31 : 12/12 0.0%
15:43:32 : Test finished.
----------
Total: 12/12 (0.0%)
Warbling and disortion on the aac encode, while faac has less harsh pre-echo but some noticable tonal disortion. However faac sounds far better.
foo_abx 1.3.4 report
foobar2000 v1.1.7
2011/06/29 15:44:06
File A: C:\Temp\Through Being Cool (ffmpeg AAC).m4a
File B: C:\Temp\Through Being Cool (ffmpeg FAAC).m4a
15:44:06 : Test started.
15:44:19 : 01/01 50.0%
15:44:24 : 02/02 25.0%
15:44:28 : 03/03 12.5%
15:44:33 : 04/04 6.3%
15:44:43 : 05/05 3.1%
15:44:49 : 06/06 1.6%
15:44:55 : 07/07 0.8%
15:44:59 : 08/08 0.4%
15:45:05 : 09/09 0.2%
15:45:13 : 10/10 0.1%
15:45:18 : 11/11 0.0%
15:45:21 : 12/12 0.0%
15:45:22 : Test finished.
----------
Total: 12/12 (0.0%)
The aac encode has horrid warbling all over the synth at the start, also it produces sharp disortion and drop outs when the lead vocals pauses. FAAC sounds better, however the vocals can produce disortion and also it struggles to cope with the recording artifacts that appear on the track.
Yep, I have tried for myself. ffmpeg AAC produces very inferior quality even at 128 kbps. It will be our low anchor.
FAAC is far superior. In fact FAAC isn't that bad at all. But there are a lot of good (commercial) AAC encoders for one test.
Current votes will be updated shortly:
http://listening-test.blogspot.com/2011/06...reparation.html (http://listening-test.blogspot.com/2011/06/discussion-on-preparation.html)
Vote for AAC test.
And I also have a question: what software uses/will use new FhG encoder?
Winamp, starting with v5.62 (released today)
Wow. Downloading right now.
Winamp, starting with v5.62 (released today)
Great news.
Plan:
Decide what to test (AAC or Multiformat) and have an idea what codecs - until July 1. It's DONE.Bitrate verification - 7-10 days - until July 7-10Sample selection - 7-10 days - until July 15-20
Checking all conditions, preparations, dummy packages etc. Developers can also take a look if everything is alright. - until July 23-24
Start of test - last week of July.
http://listening-test.blogspot.com/2011/06...reparation.html (http://listening-test.blogspot.com/2011/06/discussion-on-preparation.html)
The period of voting for AAC or multiformat public test is over.
There are more votes in favor of AAC test. The discussion on multiformat test will be continued in next year.
The list AAC encoders and first approximation of the settings for ~96-100 kbps:
1.
Nero -q 0.345
2.
QuickTime [TVBR and/or CVBR (?)] . The members suggest what settings to use during bitrate verification.
qtaacenc:
TVBR: --tvbr 46 --highest --samplerate keep
CVBR: --cvbr 96 --highest --samplerate keep or iTunes CVBR equivalent : --cvbr 96 --normal --samplerate keep
3.
FhG VBR 3 (Winamp 5.62)
4.
Coding Technologies (Winamp 5.61). In case if most of the codecs will produce around 100 kbps on average (more likely) then we can use MediaCoder. It can encode with CT at 100 kbps
Low anchor is ffmpeg AAC at 128 kbps. (very inferior quality)
People, suggestions, please.
P.S. Everybody is welcome to #hydrogenaudio on irc.freenode.net
Vote for AAC test.
And I also have a question: what software uses/will use new FhG encoder?
Winamp, starting with v5.62 (released today)
Can you make a news submission (specifically the new encoder)? This is interesting enough news.
It's important to mention that Quicktime had (or has ) a bug at 96 kbps. It appears like lowpass wasn't applied properly and there were some quality drops.
If encoded file is lowpassed at 15.7 kHz then everything is ok.
I tried nero (1.5.4), FgH (Winamp 5.62), QT (7.6.9) encoders on 2 releases: http://i.imgur.com/Xs8t6.png (http://i.imgur.com/Xs8t6.png)
That's not much, but knowing what kind of music is behind this releases I might expect that TVBR 46 would be min at average and nero max in larger set of tracks, with recommended parameters
If we consider boxplot of release bitrates as relevant, then QT TVBR seems like right choice over CVBR: http://i.imgur.com/1sVYy.png (http://i.imgur.com/1sVYy.png)
Again this are only two unrepresentative releases, (one is noise/microsound and other highly compressed d'n'b), but as I made it out of my curiosity (along with spectrograms etc ) I thought to post it here also
Thank You, romor.
That's exactly what we should do. Everybody reports his/her resulting bitrates and suggestion about CVBR/TVBR choice. It can be 1 album or whole collection.
I will create Excel table later.
My quick test:
Karl Bohm, Wiener Philharmoniker / Mozart: Requiem d-moll, KV.626
Nero 1.5.4.0 -q 0.345 -ignorelength - 96 kbps avg
QAAC v0.47 (QuickTime 7.6.9) -s -V 46 -q 2 - 99 kbps avg (QAAC's resultant audio stream is bit-identical to
QTAACENC --tvbr 46 --highest --samplerate keep)
But as for me there are also audible (and visible) diferences in encoding of high frequencies (while the lowpass filter frequencies are nearly equal). I can hear that for QT background noise has a little wider frequency range:
foo_abx 1.3.4 report
foobar2000 v1.1.7
2011/07/02 11:15:26
File A: D:\Моя музыка\Mozart\Mozart - Requiem [Mathis, Hamari, Ochman, Ridderbusch - Karl Bohm] (1990) - DG\03 Tuba mirum_qaac.m4a
File B: D:\Моя музыка\Mozart\Mozart - Requiem [Mathis, Hamari, Ochman, Ridderbusch - Karl Bohm] (1990) - DG\03. Tuba mirum_nero.m4a
11:15:26 : Test started.
11:15:58 : 01/01 50.0%
11:16:03 : 02/02 25.0%
11:16:10 : 03/03 12.5%
11:16:21 : 04/04 6.3%
11:16:27 : 05/05 3.1%
11:16:39 : 06/06 1.6%
11:16:52 : 07/07 0.8%
11:16:55 : 08/08 0.4%
11:16:59 : 09/09 0.2%
11:17:02 : 09/10 1.1%
11:17:06 : 10/11 0.6%
11:17:13 : 11/12 0.3%
11:17:20 : 12/13 0.2%
11:17:32 : 13/14 0.1%
11:17:36 : Test finished.
----------
Total: 13/14 (0.1%)
Here is the spectrogram (http://audiophilesoft.ru/misc/qttvbr_nero.jpg) of two tracks (the first part is for QT, second - for Nero).
@Steve Forte Rio,
Ok, I see QuickTime works fine on your machine (no bug).
Can I ask you to encode a few albums (different musical genres) with Nero q0345, CVBR 96, TVBR 46, Fhg VBR 3 for bitrate verification? Winamp's CT is CBR so no need to verificate it. Tell me if these settings produce similar bitrate (100 kbps) and if some of them need any correction.
Also it's important to check real file size instead of foobar's bitrate information. Sometimes it doesn't inform correctly.
As I'm (one of) conductor of the test it will be too much control for one person (me) so I shouldn't verificate bitrate. I will rely on your and other members results of bitrate verification.
Can I ask you to encode a few albums (different musical genres) with Nero, CVBR, TVBR for bitrate verification?
Ok , I'll do it a little later and will report a file-based bitrate (with tags removed) here.
Thank You.
What are the "default" settings for AAC 96kbps on Itunes OSX and Itunes Windows?
Well, here is my results of bitrate verification.
Unfortunately, the variety of genres isn't too big:
Mozart - The Late Symphonies, CD1 (symphonic)
Mozart - Requiem D-moll KV626 (choral)
Bach - French Suites, Vol.1 (solo instrumental)
Led Zeppelin - Led Zeppelin IV, Coda (rock/n/roll)
The Beatles - One (rock/n/roll)
VA - Golden Jazz (jazz)
Bee Gees - The Greatest Hits, CD2 (pop)
Armin van Buuren - Mirage (electronica/trance)
Total duration: 9:38:22 (34702 seconds)
All sources are true lossless
Duration of resultant files for all coders except Winamp FhG is identical to the original. Seems like FhG can't encode gaplessly - I've got +3 seconds totally for it.
Bitrates:
PCM WAV: (6 121 423 616 bytes / 34702 seconds) * 0.008 =
1411.2 kbit/s[TVBR] QAAC v.0.47 (QT 7.6.9) -V 46 -q 2: (422 549 458 bytes / 34702 seconds) * 0.008 =
97.4 kbit/s[CVBR] QAAC v.0.47 (QT 7.6.9) -v 96 -q 2: (441 005 435 bytes / 34702 seconds) * 0.008 =
101.7 kbit/sNero 1.5.4.0 -q 0.345 -ignorelength: (441 633 426 bytes / 34702 seconds) * 0.008 =
101.8 kbit/sWinamp 5.62 FhG VBR 3: (456 149 050 bytes / 34702 seconds) * 0.008 =
105.2 kbit/s(I guess, I must divide on original duration, right? If not, then bitrate will be 105.1 kbit/s )
So, that's all for now
Let's see how it compares with my results.
TVBR: --tvbr 46 --highest --samplerate keep.
93.62 kbpsCVBR: --cvbr 96 --highest --samplerate keep.
101.02 kbpsFhG VBR 3.
101.3 kbpsNero -q 0.345.
101.89 kbpsWinamp CT is CBR ( hence doesn't need bitrate verification).
(http://s4.postimage.org/i7dkl38zm/Bitrates_2011_AAC.png)
Excel table www.mediafire.com/?guujfe9qny2kb32
TVBR produces considerably lower bitrate. Windows version of Quicktime produces the same bitrate with --tvbr 45, 46 ...48. So there is no possibility to shift bitrates. While Mac version of QT can shift bitrates.
Unless some MAC user want to help here.
Other way we can include both modes CVBR and TVBR (but we should exclude Winamp's CT encoder then.)
Suggestions and opinions, please.
What are the "default" settings for AAC 96kbps on Itunes OSX and Itunes Windows?
iTunes CVBR equivalent : --cvbr 96 --normal --samplerate keep
Windows version of Quicktime produces the same bitrate with --tvbr 45, 46 ...48.
I noticed the same yesterday while trying to tune switches, and at the end I figured you did fine job there
I tested more releases today also:
graph: http://i.imgur.com/i17Pq.png (http://i.imgur.com/i17Pq.png)
boxplot: http://i.imgur.com/mNeRL.png (http://i.imgur.com/mNeRL.png)
csv: http://pastebin.com/t1VtpNpY (http://pastebin.com/t1VtpNpY)
from which my previous assumption seems wrong and your suggested parameters shows fine
highlight (149 tracks, 14 releases):
nero: 96
FgH: 93
QT-CVBR: 100
QT-TVBR: 94
@romor
Thank you for your work.
What exact settings do you use for QT-CVBR and TVBR?
suggested from post #76, of course
why do you ask?
TVBR: --tvbr 46 --highest --samplerate keep
CVBR: --cvbr 96 --highest --samplerate keep
This way everybody will understand what we are talking about.
Current bitrate table:
(http://s4.postimage.org/i7zznclz6/Bitrates_2011_AAC_members.png)
Also, it's interesting to see how flac and FgH bitrates correlate (in above set 0.8)
You and Steve did not provide data per track, maybe it can be noticed there as well?
You and Steve did not provide data per track, maybe it can be noticed there as well?
I just didn't want to ask too much. But everybody can do some extra job as you've done.
It's a same job, assuming flac version is source, but yeah, maybe not all find such things interesting
I don't know why images aren't displayed properly.
Current bitrate table:
http://listening-test.blogspot.com/2011/07...rification.html (http://listening-test.blogspot.com/2011/07/bitrate-verification.html)
(http://s2.ipicture.ru/uploads/20110703/1dYe5VEu.png)
My particular bitrate table
(http://s2.ipicture.ru/uploads/20110703/3apQR3et.png)
Windows version of Quicktime produces the same bitrate with --tvbr 45, 46 ...48. So there is no possibility to shift bitrates. While Mac version of QT can shift bitrates.
Mac version also produces the same bitrate. I guess you are confusing it with the --cvbr/abr/cbr switch, which accepts odd bitrates in the mac version.
You're right. I've confused it with CVBR odd bitrates from old thread http://www.hydrogenaudio.org/forums/index....st&p=685749 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77272&view=findpost&p=685749)
Then we should make choice.
We can include both CVBR and TVBR. Single TVBR mode won't be enough representative due to its lower bitrate (-6.2%). But we should exclude Winamp CT encoder.
There were a lot of interest to compare (popular) iTunes CVBR and TVBR. http://www.hydrogenaudio.org/forums/index....st&p=682691 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77272&view=findpost&p=682691)
I know we will return to this question (CVBR at 100-101 kbps vs TVBR 95 kbps) again. There is no guaranty that TVBR at 95 kbps is any better than CVBR at 100-101 kbps.
New candidate list: Nero, TVBR, CVBR, Fhg.
Votes also show that comparison between CVBR and TVBR is inevitable http://listening-test.blogspot.com/2011/06...reparation.html (http://listening-test.blogspot.com/2011/06/discussion-on-preparation.html)
Objections?
Duration of resultant files for all coders except Winamp FhG is identical to the original. Seems like FhG can't encode gaplessly - I've got +3 seconds totally for it.
Yes, unfortunately there is a bug that makes the new encoder sometimes not write the gapless data. I just fixed it. I'll post an updated DLL here shortly for everyone, and we should get the release out in about a week.
Some previous public AAC tests included 5 codecs. So it's not impossible. http://listeningtests.t35.com/ (http://listeningtests.t35.com/)
It can be: Nero, QT CVBR, TVBR, Fhg, CT.
The bitrate verification will end on 7-8 July. This way there will be more time for sample selection 8-18 July and test will be started a few days earlier (20 July)
I'm for including of 5 codecs... It would be too unpleasant to exclude TVBR or CT AAC from the test.
Yes, unfortunately there is a bug that makes the new encoder sometimes not write the gapless data. I just fixed it. I'll post an updated DLL here shortly for everyone, and we should get the release out in about a week.
Thanks
Interesting an AAC test would win over a multiformat test. Intriguingly, I guess that's where the public eye has come to rest in terms of encoding interest (if not decoding -- the ubiquity of MP3 is still with us). In any case, I was hoping GXLame-t5.2 would finally get in the running.
Yes, unfortunately there is a bug that makes the new encoder sometimes not write the gapless data. I just fixed it. I'll post an updated DLL here shortly for everyone, and we should get the release out in about a week.
Thanks
Here you go - http://www.hydrogenaudio.org/forums/index....showtopic=89487 (http://www.hydrogenaudio.org/forums/index.php?showtopic=89487)
Thanks for spotting the bug.
In any case, I was hoping GXLame-t5.2 would finally get in the running.
Why didn't you talk? I mentioned GXLame here and there.
Until now the candidate list is as following
Bitrate is 100 kbps.
1. Nero -q 0.345
2. QT-TVBR --tvbr 46 --highest --samplerate keep
3. QT-CVBR --cvbr 96 --highest --samplerate keep
4. FhG VBR 3 (Winamp 5.62)
5. Coding Technologies (Winamp 5.61). Bitrate is shifted to 100 kbps (Mediacoder).
+ low anchor ffmpeg's AAC 128 kbps.
Suggestions, please.
The selection of samples will start from 7 July.
I'm no particular interested on AAC codecs, but i see no problem with your selection.
I tested the bitrates of my usual test file sets (25 various and 25 classical tracks) using the above mentioned settings:
Various Classical All
QT CVBR 99.9 99.0 99.5
QT TVBR 94.4 94.4 94.4
NERO 102.3 92.8 97.6
WA/FhG 104.2 90.8 97.5
Regarding the Winamp/CT encoder I'd prefer to have the normal 96 kbps Winamp setting instead of the Mediacoder tweak which is not available in Winamp. It would be a quite fair comparison of the old and new Winamp encoder.
I'll upload an Excel table of the file specific bitrates and add the link here.
EDIT
The bitrate table: http://www.hydrogenaudio.org/forums/index....st&p=762129 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=89518&view=findpost&p=762129)
Current table of bitrates per member
(http://s2.ipicture.ru/uploads/20110706/MRJU9H0J.png)
Regarding the Winamp/CT encoder I'd prefer to have the normal 96 kbps Winamp setting instead of the Mediacoder tweak which is not available in Winamp. It would be a quite fair comparison of the old and new Winamp encoder.
All right.
The average bitrate of all codecs is 98.5 kbps. So it can be 98 kbps for Winamp CT. Or leave it as is (native winamp's 96 kbps).
Though let's see what other members will say. (it's a last day to suggest settings).
I'm no particular interested on AAC codecs, but i see no problem with your selection.
Thank You. Even such simple answer is useful. (instead of silence)
While there are still a few hours to make suggestion for choice of codecs/settings all members can submit their samples for this test.
Here http://www.hydrogenaudio.org/forums/index....showtopic=89518 (http://www.hydrogenaudio.org/forums/index.php?showtopic=89518)
Time slot: ~1 week
/mnt and Steve Forte Rio will conduct the process of sample selection (as we have talked via PM).
I have done a bitrate test with 40 tracks from my collection.
(http://img585.imageshack.us/img585/7676/aacbitrates96mnt.png) (http://imageshack.us/photo/my-images/585/aacbitrates96mnt.png/)
After adding some non electronic and hard rock tracks on the test, the bitrates from both Fhg and Nero are still high.
Edit: redone bitrate test.
I have done a bitrate test with 20 tracks from mixed genres.
Nero at 0.345 and V3 Fhg seems to produce higher bitrates with my selection.
I don't know how much I'd consider that song list to be "mixed genres". I've noticed the same for electronic and hard-rock music, also. In some sense, we can assume this is the encoder being smart (after all, VBR is meant to be constant-quality, regardless of input type). But I know what you are saying: this sort of bitrate variation can make it difficult to fairly compare VBR codecs.
Nero at 0.345 and V3 Fhg seems to produce higher bitrates with my selection.
Assuming that FgH is 'high' correlated with FLAC (0.8) and also Nero somewhat less (0.6) while QT totally unrelated (0.03, 0.01) from my tests, both encoders can be 'controlled' by source FLAC bitrate performance.
Or, I guess all you tracks have relatively high FLAC bitrates
Just a thought
Nero at 0.345 and V3 Fhg seems to produce higher bitrates with my selection.
Assuming that FgH is 'high' correlated with FLAC (0.8) and also Nero somewhat less (0.6) while QT totally unrelated (0.03, 0.01) from my tests, both encoders can be 'controlled' by source FLAC bitrate performance.
Or, I guess all you tracks have relatively high FLAC bitrates
Just a thought
Sounds like you might be right:
(http://img163.imageshack.us/img163/7067/mntfoobaraacbitratelayo.png) (http://imageshack.us/photo/my-images/163/mntfoobaraacbitratelayo.png/)
My foobar2000 setup displays lossless bitrates in compression ratio. The highest bitrate FLAC i had on the playlist was 'It's All About the Pentiums' and that also scores high bitrates with Nero and FhG.
Final bitrate table:
(http://s2.ipicture.ru/uploads/20110707/P1gi45j4.png)
Final version of the list with codecs and settings:
1. Nero -q 0.345
2. QT-TVBR --tvbr 46 --highest --samplerate keep
3. QT-CVBR --cvbr 96 --highest --samplerate keep
4. FhG VBR 3 (Winamp 5.62)
5. Coding Technologies (Winamp 5.61). Bitrate is shifted to 100 kbps (Mediacoder) because all other encoders (except TVBR) produce 100 kbps.
+ low anchor ffmpeg's AAC 128 kbps.
Thank You to all who have contribute to verification of bitrates.
My foobar2000 setup displays lossless bitrates in compression ratio. The highest bitrate FLAC i had on the playlist was 'It's All About the Pentiums' and that also scores high bitrates with Nero and FhG.
By "compression ratio," is this in relation to 1411kbps established by the Redbook format, or some other metric? Feel free to respond via PM if you'd rather keep this side discussion off the thread.
Regarding CT AAC encoder and 100 kbps setting: it's interesting that aacPlusCLI (enc_aacPlus.exe, commandline wrapper for enc_aacplus.dll) can produce CBR files with arbitrary bitrate... Unfortunately, it has several bugs in its code and it doesn't make 100% correct files.
I have asked Benski some time ago if it's ok to use MediaCoder to produce odd bitrates. He said it was ok.
We will check the streams later.
Unfortunately, it has several bugs in its code and it doesn't make 100% correct files.
more specifically?
I have asked Benski some time ago if it's ok to use MediaCoder to produce odd bitrates. He said it was ok.
You could also do it in winamp if you manually edit the encoder settings INI file (i think it's %appdata%\winamp\plugins\ml_transcode.ini), although the configuration UI *might* revert to a different bitrate when it gets displayed. You could uncheck "show me this every time" in the pre-transcode configuration UI to avoid that, though.
more specifically?
foobar2000 verifier complaints:
"
Warning: Reported length is inaccurate : 4:30.520000 vs 4:30.460340 decoded"
This is relatively easy to fix, but benski's advice looks better IMHO.
more specifically?
foobar2000 verifier complaints:
"Warning: Reported length is inaccurate : 4:30.520000 vs 4:30.460340 decoded"
This is relatively easy to fix, but benski's advice looks better IMHO.
If I remember right, Winamp 5.61 (with CT AAC) can't encode gaplessly too.
If I remember right, Winamp 5.61 (with CT AAC) can't encode gaplessly too.
No problems here. (Winamp 5.62 + enc_aacplus.dll from 5.61, but I don't think that it matters)
If I remember right, Winamp 5.61 (with CT AAC) can't encode gaplessly too.
I could get gapless with CT AAC files under Winamp, but not on foobar2000 and iTunes. Sadly i cannot get gapless playback working with QuickTime and Nero AAC encoded files under Winamp. Gapless support for AAC is pretty messy outside foobar2000 and iTunes and completely non-existent on Linux.
I would like to participate in this listening test with completely new AAC encoder that will be included in Easy CD-DA Extractor 15.3.
As you may be aware of; Current Easy CD-DA Extractor version uses the same Coding Technologies (v8.2.0) encoder as Winamp 5.61. This is now going to be replaced with further improved encoder in v15.3.
I am going to release the new version with the new AAC encoder before 31st July. I will be able to announce the encoder details after the weekend.
No
Ok, so there is no interest to test the Dolby Pulse encoder against Nero and Fraunhofer, or is there ?
Ok, so there is no interest to test the Dolby Pulse encoder against Nero and Fraunhofer, or is there ?
There might be interest, but the test was announced long ago, and the deadline for encoders already closed 2 weeks ago. I don't think the person pouring his time in organizing this test will want to delay or redo everything because one encoder might get a new release. Then you can keep postponing the tests forever, because by the time that's out, something else might have an update pending, too.
Ok, it is totally understandable. I was just so excited after my own experimental tests with the new encoder.
Maybe there is a chance to organize a new test sometime in the near future, maybe HE-AAC test ?
(sorry foraccidental double-posting)
Perhaps, and please realize that anyone else can conduct/host such a test, assuming he has the time and resources.
In the meantime, let's keep this on-topic, please.
@jukkap
Can you open the new topic and tell us more about upcoming Dolby Pulse AAC encoder?
You could also do it in winamp if you manually edit the encoder settings INI file (i think it's %appdata%\winamp\plugins\ml_transcode.ini), although the configuration UI *might* revert to a different bitrate when it gets displayed. You could uncheck "show me this every time" in the pre-transcode configuration UI to avoid that, though.
I have encoded the CT files this way. It works. Thank you for explanation.
Some notes:
FhG 1.01 was used for this test. http://www.hydrogenaudio.org/forums/index....st&p=761865 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=89487&view=findpost&p=761865)
Low anchor is ffmpeg's AAC but at 96 kbps http://ffmpeg.zeranoe.com/builds/win32/sta...win32-static.7z (http://ffmpeg.zeranoe.com/builds/win32/static/ffmpeg-git-5d4fd1d-win32-static.7z)
128 kbps was too good to be the low anchor. lvqcl was right about it.
It was proposed some time ago to concatenate the sample because CVBR has some not constant behaviour on some samples.
But the downside of concatenation that it's invasive for encoders. So we would solve one problem but create another ones.
I wouldn't worry about CVBR as it has produced the same bitrate distribution on the test samples as other competitors.
Here is a complete package (ABC/HR and all samples). www.mediafire.com/?q3lrkb3ogll8jdt
ABCHR configurations for verification http://www.hydrogenaudio.org/forums/index....st&p=763167 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=89518&view=findpost&p=763167)
Please, check all conditions.
If everything is good then the test will be open this weekend.
Sample 01 - Reunion Blues
Sample 02 - Castanets
Sample 03 - Berlin Drug
Sample 04 - Enola Gay
Sample 05 - Malher
Sample 06 - Toms Diner
Sample 07 - I want to break free
Sample 08 - Skinny2a
Sample 09 - Fugue Premikres notes
Sample 10 - Jerkin Back n Forth
Sample 11 - Blackwater
Sample 12 - Dogies
Sample 13 - Convulsion
Sample 14 - Trumpet
Sample 15 - A train
Sample 16 - Enchantment
Sample 17 - Experiencia
Sample 18 - Male speech
Sample 19 - Smashing Pumpkins - Earphoria
Sample 20 - on the roof with Quasimodo
Thanks You very much , /mnt and Steve Forte Rio, for your collaboration on random selection of the samples.
Sample 01 - Reunion Blues
Where does that come from? In the Samples_selected.xls posted by Steve Forte Rio (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=89518&view=findpost&p=762958), there was a "1.flac" which seems to have been replaced.
Not that I care much, but what was the reason for this change?
And another kind request: Please check for artifacts in the 20 Winamp encodings which are similar to the one Alex B described (http://www.hydrogenaudio.org/forums/index.php?showtopic=89487&st=0&p=762274&#entry762274). These reveal a (rare) bug. I did a brief test and couldn't find any, so that bug doesn't seem to show up in the test samples, but I'm sick at the moment, and my hearing sucks.
Thanks,
Chris
The samples are the same.
Reunion Blues is ''Jazz''
And "1.flac" is Smashing Pumpkins - Earphoria
And another kind request: Please check for artifacts in the 20 Winamp encodings which are similar to the one Alex B described (http://www.hydrogenaudio.org/forums/index.php?showtopic=89487&st=0&p=762274&#entry762274). These reveal a (rare) bug. I did a brief test and couldn't find any, so that bug doesn't seem to show up in the test samples, but I'm sick at the moment, and my hearing sucks.
I haven't found any issues. Some loud samples like 7,8 and 17 were suspicious at the first time but I couldn't hear anything wrong with them.
Sample "Skinny2a" is Grave Wisdom by Skinny Puppy.
Reunion Blues is ''Jazz''
And "1.flac" is Smashing Pumpkins - Earphoria
I haven't found any issues.
Great, thanks!
Chris
Igor, I was just asking two simple questions. What is the problem? You were asking to "Please, check all conditions.", so I did a bit of that.
Chris
@jukkap
Can you open the new topic and tell us more about upcoming Dolby Pulse AAC encoder?
I will be able to provide more information at the time the software is actually released.
Hm, I can't see a real good reason why FhG encoder shouldn't participate.
Though Chris' questions probably have in mind giving 'his' encoder good chances I can't see a real inappropriate behavior.
In case you find it inappropriate to answer the question for a change in samples (not my opinion), you just could refuse the answer.
I'd find it great if you could give FhG another chance.
Igor, I think your decision was overkill. FhG encoder is one of the most interesting participants in this test. Could you give it another chance?
Chris, I understand your interests here (particularly your concern over the samples selection).
This test is public and fully transparent for everyone. But you have to agree - this peculiarity mustn't negatively affect it's success.
Hopefully, we'll settle this conflict as soon as possible and will return to our independent testing.
IgorC, final warning. Either you continue the conversation and the test in a civilized manner, or you are on your way out here.
Fine. Understood.
I'd find it great if you could give FhG another chance.
I think your decision was overkill. FhG encoder is one of the most interesting participants in this test. Could you give it another chance?
In fact I'm also very interesting to test it. So it's in.
I hope all previous disagreements were just misunderstanding (not English native speakers) .
I don't think it's a waste of time to test FAAC, by far most AAC content is encoded with it, especially in the 96kbps range. Testing FAAC can provide clear and up to date argumentation for why many internet services should or shouldn't keep using a free solution as opposed to a commercial solution.
Hi menno, long time no see. Could you tell me which internet services use FAAC at 96 kbps (YouTube within the FLV/MP4 format?) or maybe send me a PM to avoid off-topic postings in this thread?
Until the beginning of the test I want to say how it's important for me. A lot of people (including me) have waited a lot of time for such test.
We will test commercial LC-AAC encoders and it will be my responsibility to conduct it in fair and open manners.
I expect the honest attitude from all interested parties and participants. We will control the results.
All you have to do is rank the competitors basing on what you hear
I will ask administrators to close this topic. The test will begin shortly.