Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multi-Codec Listening Test: 96-128-192-256Kbps (Read 61728 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #50 –
DigitalDictator:
I didn't get what was your problem, but it doesn't matter anyway the problem was solved thks to rpp3po & I don't plan to test Itunes AAC anymore.
If you think Itunes ACC is great, just use it ... I have no problem with it. I didn't test its true VBR mode so maybe it is really great... I don't know & I don't care.
I didn't confuse anything, it's Itunes for windows which isn't clear/friendly at all. It doesn't even tell you that you are using the constrained VBR mode, how could I have known ?
Don't be such a ... Dictator

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #51 –
Quote from: sauvage78 on
I tried to catch something but I failed ... can you tell me when & what to listen to more exactly, I focused on synth at the beginning/middle & end, I found nothing.


Well, now I listen it back myself, it isn't that clear anyway.  Attached: the exact problem and two vorbis files at q0 and q2, as the title says
Music: sounds arranged such that they construct feelings.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #52 –
I think that I catched what you meant, there is something, but I don't think that this is really a killer sample, it is just a sample that doesn't archive transparency at low bitrate, I would rate it light/yellow artefact at Q0 so I guess that at higher bitrate I will be unable to ABX it. I am searching for samples that have problems at 128-192Kbps more than at 96Kbps because I expect that LOTS of samples will not be transparent at 96Kbps. There is much worse problem cases around, so honestly I doubt that I will add it, thks anyway. If it's not transparent at 192Kbps then it's most likely a good custumer for the job.

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.6.3
2009/03/22 16:29:11

File A: C:\Documents and Settings\SolidInc__Artefact__Lossless.flac
File B: C:\Documents and Settings\SolidInc__Artefact__Lossless_q0.ogg

16:29:11 : Test started.
16:30:11 : 01/01  50.0%
16:30:52 : 02/02  25.0%
16:31:22 : 03/03  12.5%
16:33:12 : 04/04  6.3%
16:34:43 : 05/05  3.1%
16:35:09 : 05/06  10.9%
16:37:06 : 06/07  6.3%
16:38:02 : 07/08  3.5%
16:38:33 : Test finished.

 ---------- 
Total: 7/8 (3.5%)


Codecs have been so tuned for it that I think that even castanets is not a real killer sample anymore. It is just a hard to encode sample that shows that you shouldn't expect transparency at low/mid bitrate. Before this test, my signature was aoTuV -q4, I had to rise my bitrate because I was a little too enthousiastic ...

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #53 –
Ah, well, I'll try to find better ones then
Music: sounds arranged such that they construct feelings.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #54 –
Quote from: sauvage78 on
DigitalDictator:
I didn't get what was your problem, but it doesn't matter anyway the problem was solved thks to rpp3po & I don't plan to test Itunes AAC anymore.
If you think Itunes ACC is great, just use it ... I have no problem with it. I didn't test its true VBR mode so maybe it is really great... I don't know & I don't care.
I didn't confuse anything, it's Itunes for windows which isn't clear/friendly at all. It doesn't even tell you that you are using the constrained VBR mode, how could I have known ?
Don't be such a ... Dictator

What? What did I do now?? Someone said he didn't like non free codecs. So I just pointed out that AFAIK they're all free. It's just that some applications (i.e. the interface), which perhaps enable more features, aren't free. So I don't see why you are upset. You tested iTunes, for that I thank you.

Well yea, I might be a dictator, but not here at HA, I'm not, haha.
//From the barren lands of the Northsmen

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #55 –
I just added the Abfahrt Hinwil sample (also known as eig), this one is very bad for Lame MP3 which is affected no matter the bitrate. Vorbis also have a problem that sounds different at Q2 with it.
This sample leave Nero AAC alone on the top of the mountain. Also it shows that depending on the sample Musepack Extreme may be better than Lame V2 (only at high bitrate). So it gives some credit to the Musepack sect out there.

I tested dozen of killer samples the problem is that either I can't ABX them or they only affect a single codec & not vorbis. I am still searching for something that I could ABX with Nero Q0.55 (I tested the Fear Factory sample but I couldn't ABX it ...)

Itunes files are not available anymore for download as I am reaching the upload limit, I have keeped it on the table for this time, but I may remove it next time. (The old results will still be available within the .odt anyway)

Errata: after I discovered that I could ABX aotuv -Q4 easyly on Abfahrt Hinwil this morning (must have been tired yesterday) I decided to downgrad it from failed to orange, -Q6 switched from X to failed (which doesn't change anything, it's still transparent for me), then I compared aotuv -Q4 & nero Q0.35, I decided to downgrad Nero Q0.35 from yellow to orange in order to be fair with aotuv. The artefact are differents, I am more sensible to echo than snapping, but once you catch it, snapping is more annoying than echo & once trained both are easy to ABX (snapping is masked within the little explosions). I add hesitations to put red for Musepack Thumb on this sample, it's only orange because Lame & Vorbis are worst at ~96Kbps, all the 3 are very bad but Lame & Vorbis are hurting while with Musepack it's just a big echo. The notation is also done by comparing each sample between codecs in order to be as fair as possible.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #57 –
If ever someone upload it I will try ...

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #58 –
Quote from: sauvage78 on
Also it shows that depending on the sample Musepack Extreme may be better than Lame V2 (only at high bitrate). So it gives some credit to the Musepack sect out there.

Amen

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #59 –
I guess I found the not just Nero AAC problem emese shadowking wrote about.
Upload doesn't work (not sufficient space).

For a limited time I put it on my webspace: emese sample
lame3995o -Q1.7 --lowpass 17

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #60 –
Thks ! I have it  I'm gonna get crazy listening to this, it loops "the human blood burns" in french  with a frightening voice in the background

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #61 –
That Abfahrt sample just rapes poor MP3. 8/8 at 320kbps.

8/8 on Kraftwerk and Rush using aoTuV -q8 as well, though both differences were subtle. Failed to scale up to -q10.

This is fun. I haven't ABXed anything in a while. Thanks for the opportunity.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #62 –
Quote from: ktf on
Quote from: sauvage78 on
I tried to catch something but I failed ... can you tell me when & what to listen to more exactly, I focused on synth at the beginning/middle & end, I found nothing.


Well, now I listen it back myself, it isn't that clear anyway.  Attached: the exact problem and two vorbis files at q0 and q2, as the title says


To come back to this sample, I played around with it and found out that it isn't really a problem for vorbis (transparant to me @ 104kbps, q 2,5) but it is a problem with Nero's AAC (transparant at q0,43 but not at q0,42 to me so around 160 kbps) and with LAME, which is only transparant at -V2 or above, 220kbps. The artifact isn't present at all in Musepack, not even at quality 1 (that's below thumb), but then the rest of the song sounds horrible...

Just in case anyone would be interested 
Music: sounds arranged such that they construct feelings.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #63 –
I just added a partial result for aoTuV exp-bs1, even incomplete it took me almost 3 hours (in case some people don't realize how even such a little test is time greedy ...), the good news is that for me the Rush sample is completely fixed, the bad news is it's the only sample affected by aoTuV  exp-bs1. I didn't noticed any progression/regression on any other sample (including the 2 samples I didn't strictly ABXed & that are greyed in the table). So aoTuV exp-bs1 might be a good fixe on some problems but it is not a magic solution. There is no "overall quality progression", it's more a targetted patch that achieve its goal on a specific sample. I was a little desapointed because the Rush sample was the first sample I tested so I was very excited in the beginning as I had some hope that if all those samples are affected by the same block switching issue & the Rush sample was fixed , then maybe it would improve all the other samples ... it wasn't the case. Still I like aoTuV exp-bs1, because I like Rush as a band & specially New World Man as a song

It is very likely that Castanet & Harlem results would be the same for aoTuV exp-bs1 & aoTuV Beta5.7, quick non-blind listening test sounded this way.

In the end, it doesn't make Vorbis shine but it's a good step to getting -q8 transparent for everyone again, only Krafwerk remains easy to ABX at -q8.

Errata2:
Because I failed to ABX Autechre aoTuV exp-bs1 -q8 & because aoTuV Beta5.7 -q8 was a yellow result that might have been either lucky guess or a succesfull ABX but taking all necessary time to succed (which mean VERY long). I upgraded aoTuV Beta5.7 -q8 from Yellow/Light to Green/Failed. I can't tell a difference between aoTuV Beta5.7 -q8 & aoTuV exp-bs1 -q8, so it was illogic to up the first in yellow & the secong in green, just because I didn't took the same time ABX both. In the same time I downgraded aoTuV Beta5.7 -Q6 from orange to red, the red artefact from -Q2 & -Q4 is still here & even if lighter, it doesn't decrease much until -q8, so there is no reason why it wouldn't be red. It's a one result up for one result down trade & this way the table highlight the artefact even brighter IMHO.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #64 –
3/8, Abfahrt, LAME 3.97 -V0 (236kbps). 

Any suggestions? (besides counting my blessings?)

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #65 –
Quote from: Axon on
3/8, Abfahrt, LAME 3.97 -V0 (236kbps). 

Any suggestions? (besides counting my blessings?)

Be glad that your that not sensitive to precho, IMO its more annoying then warbling artifacts.

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.6.4
2009/04/15 03:32:35

File A: C:\Downloads\01__DCT_Killer_Samples__Lossless_\01- DCT Killer Samples (Lossless)\02- Artefact Only\01- Abfahrt Hinwil (Artefact Only) Lossless.flac
File B: C:\Downloads\04__Lame_3.98.2\04- Lame 3.98.2\01- Abfahrt Hinwil\01- Abfahrt Hinwil (Artefact Only) Lame 3.98.2 V0.mp3

03:32:35 : Test started.
03:32:45 : 01/01  50.0%
03:32:52 : 02/02  25.0%
03:32:57 : 03/03  12.5%
03:33:05 : 04/04  6.3%
03:33:10 : 05/05  3.1%
03:33:22 : 06/06  1.6%
03:33:28 : 07/07  0.8%
03:33:36 : 08/08  0.4%
03:33:43 : 09/09  0.2%
03:33:51 : 10/10  0.1%
03:33:57 : 11/11  0.0%
03:34:02 : 12/12  0.0%
03:34:08 : Test finished.

 ---------- 
Total: 12/12 (0.0%)

The beats makes cracking noises. IMO eig sounds less annoying at V0, while the 1980's dance club track from RoboCop has some really bad precho issues that sound really bad even at 320 kbps.
"I never thought I'd see this much candy in one mission!"
 

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #66 –
Quote from: Axon on
3/8, Abfahrt, LAME 3.97 -V0 (236kbps). 

Any suggestions? (besides counting my blessings?)

My suggestion is to listen to the eig (aka Abfahrt Hinwihl) result of other mp3 encoders. Try for instance previous Lame versions (EDITED: I just saw you did. Try 3.98 for a comparison) or a recent Fraunhofer encoder.
You'll find that Lame 3.98.2 does a very good job within the restrictions of mp3.

The sample just shows that mp3 pre-echo can be very bad. Like in this case the really bad problems are expected to come from electronic music because impulses can be arbitrarily artificial here.
With music originating from natural instruments things usually are fine or at least not very bad even with mp3.

Lovers of electronic music may be happier using another codec.
lame3995o -Q1.7 --lowpass 17

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #67 –
Also the PTP samples also shows how bad MP3 can be with electronic music.
"I never thought I'd see this much candy in one mission!"

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #68 –
I just added results for Celt 0.52, due to space restriction I cannot provide the encoded files (it's decoded WAV packed in flac) so only the logs are available. IMHO you cannot draw any conclusion at this early stage of development, it sounds bad but it's experimental non-frozen code so I didn't expected it to sound good, I was already very happy that Xiph provided a win32 build so that I could test it. I was also happy that it was not affected at all by 2 samples. Other than that it is absolutly un-tuned & unusable for backup actually. This is only a test for audio enthousiasts.

Note: The artefact noted as medium for Celt are often severe medium, I could have noted them as M+ instead of just M, but I marked them as medium because if I wouldn't have done so, almost everything would be in the red zone. Actually Celt 256Kbps is clearly worst than Vorbis 128Kbps, I am not even sure that it can compete with Vorbis 96Kbps.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #69 –
Quote from: sauvage78 on
PS:
(...)
If ever Gabriel or Roberto reads this plz fix http://lame.sourceforge.net/quality.php (I get a 403 error when I try to download samples)


I just saw you can download them "by hand". Just paste the displayed name behind the link, http://lame.sourceforge.net/download/samples/track7.wv for instance. Only The first one, labeled Roel's infamous velvet.wv, won't work like that.
Music: sounds arranged such that they construct feelings.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #70 –
Thks for the tip ktf 

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #71 –
I have tested all the samples in the lame archive at aotuv -q4 & lossywav -q 1.5:

I can ABX applaud, spahm & testsignal3 at aotuv -q4 & below
I can ABX Fools at lossywav -q 1.5 & below

applaud reminds me of harlem
testsignal3 reminds me of kraftwerk but not as bad

spahm is more specific, I tested all this 3 samples on aoTuV exp-bs1 with the hope that maybe spahm would be improved but exp-bs1 has no affect (except on Rush)

Thks again ktf, the Fools sample will be very usefull for lossywav tuning.

If someone have information on artist album track ... for applaud, spahm & testsignal3 I am interested (I have all usefull informations for the fools sample), when exp-bs1 will be released I will remove the Rush sample from the test has it will be fixed, I may add one of these 3 samples (applaud is unlikely has it would be a doublon with harlem, I also still have Ministry & Pierre Henry samples in my bag but those are not targetting vorbis)

I now have almost all the samples I searched except one called Amnesia (acid music) I heard from Pio2001, if ever someone have this one.

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #73 –
Thks too Alex B, I have quickly tried these two (plus Track02cut from the same topic) on aotuv -q4 & lossywav -q 1.5:

I can ABX amnesia on aotuv -q4, I failed to catch anything on all the other samples/parameters. Will retry once later today then I'll give up.

Edit: Attached Fool's Garden for lossywav testing (This is a temp link for Nick.C & co)

Multi-Codec Listening Test: 96-128-192-256Kbps

Reply #74 –
Quote from: sauvage78 on
I just added results for Celt 0.52, due to space restriction I cannot provide the encoded files (it's decoded WAV packed in flac) so only the logs are available. IMHO you cannot draw any conclusion at this early stage of development, it sounds bad but it's experimental non-frozen code so I didn't expected it to sound good, I was already very happy that Xiph provided a win32 build so that I could test it. I was also happy that it was not affected at all by 2 samples. Other than that it is absolutly un-tuned & unusable for backup actually. This is only a test for audio enthousiasts.

Note: The artefact noted as medium for Celt are often severe medium, I could have noted them as M+ instead of just M, but I marked them as medium because if I wouldn't have done so, almost everything would be in the red zone. Actually Celt 256Kbps is clearly worst than Vorbis 128Kbps, I am not even sure that it can compete with Vorbis 96Kbps.


Awesome!  Thanks for testing.  Is there any chance you could do a quick check of CELT with your "killer samples" in mono mode?    Stereo support in CELT 0.5.2 is pretty much entirely untuned (and the stereo infrastructure will likely be entirely replaced in the next release).  The current version also has some internal limitations at high rates removing them is on my todo list, and they may be making 256 perform no better than 192. Transparency under critical testing at high bitrates (your backup use case) hasn't yet been an area of active tuning, but it is something that we want to get right. 

If you have the patience to do a little more testing than just a single mono check I could build a special encoder/decoder that exposes some internal settings as knobs that you could try tuning.  It seems like your samples are largely block switching stress test cases.  The block switching logic in CELT is pretty simplistic and also pretty much untuned. On the other hand, CELT's long blocks are the size of typical Vorbis shorts, so the switching doesn't need to be too smart (CELT only has shorts at all because without them AAC LD was clearly outperforming it on castanets; likely due to TNS).