Skip to main content

Topic: Vorbis quality – wrong direction? (Read 43033 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • tigre
  • [*][*][*][*][*]
Vorbis quality – wrong direction?
Reply #25
Quote
EDIT:  I've also done the same process with guruboolez' harpsichord sample (uploaded to the same thread). Looking at the spectrum (I know, its naughty  ), the HF difference is mostly above 8 kHz.  The spectrum of the violin sample also has most of this high frequency difference above 8 kHz.

Hopefully something useful can be gathered from this.

Not much, I think. If the substraction shows content in > 8kHz range only this only tells you that both encodes are different in this range, while they're very similar/identical below 8kHz, but you don't know if one encoder contains more energy in >8kHz band than the other. The difference could simply be caused by phase shift (probably oversimplified, but to get the idea)

You could gather more useful information from  (CEP/Audition) frequency analysis of identical positions.
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • music_man_mpc
  • [*][*][*][*][*]
  • Members (Donating)
Vorbis quality – wrong direction?
Reply #26
Quote
I encoded guruboolez' violin sample with the standard 1.0.1 coder and nyaochi's uncoupled stereo coder.  Then in CoolEdit, I did a mix paste and saved the differences in order to (informally) highlight the differences.

I don't think Dibrom would be too impressed with this stuff.  Maybe you should back it up with a listening test of you own?  I just don't want to see you get flamed. 
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #27
Quote
I don't think Dibrom would be too impressed with this stuff.  Maybe you should back it up with a listening test of you own?  I just don't want to see you get flamed.  

The point of the exercise is to pinpoint the differences that uncoupled stereo has over coupled stereo rather than to compare coders subjectively (ie. this coder sounds better than that one).  I'm not making any claims here, but just doing a bit of detective work and seeing if stereo coupling really is the cause of HF boost in Vorbis.

I know it's bad (I even stated it that this is bad) but I don't have good ears and have trouble hearing these distortions so I need an empirical way of observing the differences that guruboolez and [proxima] are noticing in their listening tests.  Hopefully Dibrom will pardon me on this. 

  • kjoonlee
  • [*][*][*][*][*]
Vorbis quality – wrong direction?
Reply #28
Will it be possible to "isolate" the HF boost?

Will it be possible to "add" the isolated HF boost to the uncompressed original?

Will the "spiked" original be ABXable against the original?

Pardon me if these are stupid questions, please..

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #29
Quote
Will the "spiked" original be ABXable against the original?

Pardon me if these are stupid questions, please..


It's quite a good idea actually.  What I might do is take these differences, add them back to the original, and get some people with good ears to ABX them.

EDIT:  I've uploaded three files where I have added the differences back to the original.  The expectation is these will be ABXable with similar distortions.
  • Last Edit: 11 February, 2004, 09:55:30 AM by QuantumKnot

  • music_man_mpc
  • [*][*][*][*][*]
  • Members (Donating)
Vorbis quality – wrong direction?
Reply #30
Quote
The point of the exercise is to pinpoint the differences that uncoupled stereo has over coupled stereo rather than to compare coders subjectively (ie. this coder sounds better than that one).  I'm not making any claims here, but just doing a bit of detective work and seeing if stereo coupling really is the cause of HF boost in Vorbis.

I know it's bad (I even stated it that this is bad) but I don't have good ears and have trouble hearing these distortions so I need an empirical way of observing the differences that guruboolez and [proxima] are noticing in their listening tests.  Hopefully Dibrom will pardon me on this. 

Fair enough.  I suppose Guruboolez will provide the conclusive evidence in the form of a listening test soon anyway, I guess its nice to have some idea of what's going on in the mean time.  Although, it still might not go over too well.
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #31
Quote
Fair enough.  I suppose Guruboolez will provide the conclusive evidence in the form of a listening test soon anyway, I guess its nice to have some idea of what's going on in the mean time.  Although, it still might not go over too well.

My choices are rather limited unfortunately and I'm unable to go into the Vorbis source code and try to tackle this HF boost problem by relying on subjective comments on quality only.

But we'll try kjoonlee's suggestion and see how it goes.  If these corrupted samples are ABXable and distortions are similar in nature to the HF boost, then we've at least verified experimentally that stereo coupling has problems 

  • [proxima]
  • [*][*][*]
Vorbis quality – wrong direction?
Reply #32
Quote
It's quite a good idea actually.  What I might do is take these differences, add them back to the original, and get some people with good ears to ABX them.

I think this is not useful because, in this manner, we are completely ignoring masking. I see no reason to abx a such sample. Maybe someone more expert can confirm my assumption.
I remember some Musepack spectral analysis where original and encoded differ a lot visually, but all the quantization noise is perfectly masked and the files are not ABXable.
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #33
Quote
,Feb 12 2004, 01:02 AM] I think this is not useful because, in this manner, we are completely ignoring masking. I see no reason to abx a such sample. Maybe someone more expert can confirm my assumption.
I remember some Musepack spectral analysis where original and encoded differ a lot visually, but all the quantization noise is perfectly masked and the files are not ABXable.

I sort of guessed it wouldn't be so simple. 

Oh well, in that case, I give up on the HF boost problem.  I guess there is little point in me trying to fix something I cannot hear that well.

I'll focus on the tonality problem in the Arche I sample which I can actually hear as well as pre-echo issues.

  • kjoonlee
  • [*][*][*][*][*]
Vorbis quality – wrong direction?
Reply #34
Indeed, I was a little hesitant to ask my questions because I don't understand noise masking very well, if at all.

Also, on second thought, I doubt if my suggestion would be very useful.

I'm sorry for the confusion.

  • tigre
  • [*][*][*][*][*]
Vorbis quality – wrong direction?
Reply #35
Quote
I sort of guessed it wouldn't be so simple.  

Oh well, in that case, I give up on the HF boost problem.  I guess there is little point in me trying to fix something I cannot hear that well.

Before you give up - have you tried using frequency analysis for comparing orignal vs. different encodes hf boost-wise, as I suggested in my last post?
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • nyaochi
  • [*][*][*]
Vorbis quality – wrong direction?
Reply #36
Let me add two samples available here. You can hear boosted close hi-hat cymbal in the 8823 sample with 1.0.1 -q4. The second sample is white noise created by CoolEdit. I suppose some of you disagree with the use of artificially created sample, but the problem with 1.0.1 -q4 is too obvious for me. I found the sample encoded by 1.0.1 -q4 totally different from the original. I'm not sure this sample exposes HF boost problem (or another problem?), but I believe it's a good material to improve Vorbis.

Actually, I created my "Modest Tuning" to improve these samples. Here's a part of my personal listening result (in Japanese only) of 8823 sample (I focused on only HF boost) with Modest Tuning beta2 (mtb2), aoTuV alpha2 (at2), GarfTuned3 beta1 (gt), QKTune beta2 (qk2).

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: 8823

1L = C:\users\musics\test\compare\8823-q5qk2.wav  (171kbps)
2L = C:\users\musics\test\compare\8823-q5at2.wav  (158kbps)
3L = C:\users\musics\test\compare\8823-q5gt.wav   (178kbps)
4L = C:\users\musics\test\compare\8823-q4mtb2.wav (169kbps)

---------------------------------------
General Comments:
---------------------------------------
1L File: C:\users\musics\test\compare\8823-q5qk2.wav
1L Rating: 3.0
1L Comment:
---------------------------------------
2L File: C:\users\musics\test\compare\8823-q5at2.wav
2L Rating: 3.5
2L Comment:
---------------------------------------
3L File: C:\users\musics\test\compare\8823-q5gt.wav
3L Rating: 2.5
3L Comment:
---------------------------------------
4L File: C:\users\musics\test\compare\8823-q4mtb2.wav
4L Rating: 4.5
4L Comment:
---------------------------------------
ABX Results:
Original vs C:\users\musics\test\compare\8823-q5qk2.wav
   13 out of 14, pval < 0.001
Original vs C:\users\musics\test\compare\8823-q5at2.wav
   13 out of 14, pval < 0.001
Original vs C:\users\musics\test\compare\8823-q5gt.wav
   13 out of 14, pval < 0.001
Original vs C:\users\musics\test\compare\8823-q4mtb2.wav
   12 out of 21, pval = 0.332


I haven't test these samples with uncoupled stereo but I'll later. 
  • Last Edit: 11 February, 2004, 12:51:41 PM by nyaochi

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #37
Quote
Quote
I sort of guessed it wouldn't be so simple.  

Oh well, in that case, I give up on the HF boost problem.  I guess there is little point in me trying to fix something I cannot hear that well.

Before you give up - have you tried using frequency analysis for comparing orignal vs. different encodes hf boost-wise, as I suggested in my last post?

No.  It was 1 am in the morning and I was getting really tired so I gave up trying.

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #38
Quote
Let me add two samples available here. You can hear boosted close hi-hat cymbal in the 8823 sample with 1.0.1 -q4.

My ABXing of the uncoupled stereo hi-hat cymbal at q 4 though I thought the differences were so subtle, I often doubted myself in most cases.

Code: [Select]
-------------------------------------
WinABX v0.23 test report
02/12/2004 10:42:23

A file: E:\vsamples\8823.wav
B file: E:\vsamples\uncoupled.wav (155.27 kbps)

10:43:57    1/1  p=50.0%
10:44:11    2/2  p=25.0%
10:44:25    3/3  p=12.5%
10:44:34    3/4  p=31.2%
10:44:47    4/5  p=18.8%
10:45:33    5/6  p=10.9%
10:45:45    5/7  p=22.7%
10:46:06    6/8  p=14.5%
10:46:34    7/9  p= 9.0%
10:46:40   8/10  p= 5.5%
10:46:57   9/11  p= 3.3%
10:47:05  10/12  p= 1.9%
10:47:11  11/13  p= 1.1%
10:47:17  12/14  p= 0.6%
10:47:24  13/15  p= 0.4%
10:47:40  14/16  p= 0.2%
10:47:47  test finished


That is quite a nice sample for HF boost.

  • kwanbis
  • [*][*][*][*][*]
  • Developer (Donating)
Vorbis quality – wrong direction?
Reply #39
i hope you all get together an make vorbis the great encoder it was meant to be ... even if forking grom xiph ... the world NEEDS a free encoder ... thank you guys
  • Last Edit: 12 February, 2004, 05:18:55 AM by kwanbis

  • ErikS
  • [*][*][*][*][*]
Vorbis quality – wrong direction?
Reply #40
Quote
,Feb 11 2004, 11:02 PM] I think this is not useful because, in this manner, we are completely ignoring masking. I see no reason to abx a such sample. Maybe someone more expert can confirm my assumption.
I remember some Musepack spectral analysis where original and encoded differ a lot visually, but all the quantization noise is perfectly masked and the files are not ABXable.

That's why you add in the original signal to the difference. That way you have the masker signals present at those time the difference signal would be masked. Just listening to the difference signal however would have the problem you mention. Right? But why not compare the uncoupled stereo with the "joint stereo" one directly. All other variables are constant right?

  • knik
  • [*]
  • Developer
Vorbis quality – wrong direction?
Reply #41
A listening may not help to fix the HF boost. I'm affraid one would need to dig deeply into the source until finds where the energy difference comes from.

  • [proxima]
  • [*][*][*]
Vorbis quality – wrong direction?
Reply #42
Quote
A listening may not help to fix the HF boost. I'm affraid one would need to dig deeply into the source until finds where the energy difference comes from.

Maybe in this case listening cannot help to resolve codec problems, we need a capable person who can understand the code and correct this problem. Anyway, due to the availability of hardware Vorbis players, i think that testing is still important because :
- maybe we can suggest the newbies a best, "HA approved" version/128 kbps settings.
- there is the chance to reduce HF boost with "small hacks" such as disabling stereo coupling.
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
Vorbis quality – wrong direction?
Reply #43
I’m late, but I didn’t have much free-time this week (and internet access is a second problem too). I have performed the test yesterday, comparing:
- RC3 (march 2002)
- 1.01 (october 2003)
- lossless coupling (Quantum Knot)
- uncoupled encoder (Nyaochi)

According to the fact that uncoupled encodings at –q4 are a lot bigger than traditionnal vorbis encoder at the same setting, I also add an uncoupled encoder at lower setting. Nyaochi uncoupled at –q2 is close to average bitrate of 1.01/RC3 –q4, but slightly inferior. I’ve therefore tested fully uncoupled encoder at –q2,3 in order to measure the negative impact of a sub-efficient (uncoupled) channel coupling at ~128 kbps, compared to the same encoder, helped with channel coupling.



Results – /|\ Don’t care about notation absolute signification: it isn’t ITU compliant





• uncoupled encoders are winning again, without any doubts.
• full uncoupled encoder [FU] (Nyaochi) is clearly superior to lossless coupled [LC] (provided by Quantum Knot) vorbis encoder. FU was always better than LC, except on one sample  – Mars: same notation. These results confirm [proxima]’s conclusions.
• both uncoupled encoders are still suffering from noise or tonality difference. The problem is not fully corrected: there are still differences. It’s not awfully annoying, but it’s difficult to forgive at ~160 kbps...

• at ~128 kbps, 1.01 appears again as a complete loser, compared to RC3 (harpsichord is the exception again) and to FU –q 2,3. There are so much noise and imprecision with 1.01... In other words, 1.01 without coupling was always better than 1.01 with coupling!!!
• But FU –q 2,3 is far from perfection: there are much more noise than with FU at –q 4. Noise isn’t the only difference. Other artifacts appeared (for example, acidity on Spagna, distortions on Brahms). And pre-echo reached an annoying level.
• Therefore, there are no clear winner between RC3 –q4 and FU –q 2,3. On four samples, RC3 sounded better; on the four others, it was FU. Overall notation is    in favor of RC3, but the difference is distorted by an approximate notation (with deliberately exaggerating contrasts).
  • Last Edit: 29 December, 2005, 04:18:00 PM by guruboolez

  • rjamorim
  • [*][*][*][*][*]
Vorbis quality – wrong direction?
Reply #44
I can only imagine the discussions that will arise when choosing codecs for the 128kbps multiformat test

Vorbis 1.0.1 vs. GT3 vs. QK vs. Nyaochi vs. CVS...
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
Vorbis quality – wrong direction?
Reply #45
Quote
I can only imagine the discussions that will arise when choosing codecs for the 128kbps multiformat test

Vorbis 1.0.1 vs. GT3 vs. QK vs. Nyaochi vs. CVS...

Yes, probably.
But you could request to people demanding for x or y encoder a blind test: nobody will bother you anymore

I guess that uncoupled encoder can't be considered as real competitors. There were released in order to isolate a problem (hiss), and probably need tuning in order to increase their efficiency.

P.S. Thank you for your private messages (I'll answer you later)

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #46
Again, thanks to guruboolez for his testing.

The aim of testing lossless and uncoupled stereo is to probe the behaviour of the HF boost.  I think from guruboolez and [proxima]'s tests, we can conclude that the lossy coupling (point) is having problems but is not the sole cause. We also see that uncoupled stereo is not perfect and still suffers from noise. 

So perhaps, the uncoupling has only reduced the effect of the hiss but not totally removed it.  That is, it hasn't made the problem go away or solved it, but only made it harder to detect.  Hence I'm beginning to believe that we can rule out stereo coupling as the cause of HF boost.  Digging into the source code of rc3 may provide new insight as well as nyaochi's focus on the noise companding in his 'Modest Tuning'.

I know there may be some other people on the Vorbis-dev mailing list who are saying 'I told you so', but hey, it never hurts to do some comprehensive listening tests to verify it.
  • Last Edit: 14 February, 2004, 07:06:58 PM by QuantumKnot

  • QuantumKnot
  • [*][*][*][*][*]
  • Developer
Vorbis quality – wrong direction?
Reply #47
I just came across results from these listening tests, performed by nyaochi.  The big table in the middle of the page should be self explanatory.

http://nyaochi.cocolog-nifty.com/audio/2004/02/128kbps.html

  • harashin
  • [*][*][*][*]
Vorbis quality – wrong direction?
Reply #48
Quote
I just came across results from these listening tests, performed by nyaochi.  The big table in the middle of the page should be self explanatory.

http://nyaochi.cocolog-nifty.com/audio/2004/02/128kbps.html

Interesting. He also releases the 1.0.1 Modest Tuning beta 2 used in these tests.
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/

  • nyaochi
  • [*][*][*]
Vorbis quality – wrong direction?
Reply #49
Quote
I just came across results from these listening tests, performed by nyaochi.  The big table in the middle of the page should be self explanatory.

http://nyaochi.cocolog-nifty.com/audio/2004/02/128kbps.html

Some notes for the test. I ranked my favorite 3 samples as well as 8 classical samples presented by guruboolez. The main purpose of this test is to evaluate aoTuV b1 -q4 compared with 1.0.1 official -q4, RC3 (March 2002) -q4, lossless coupling -q4, uncoupled stereo -q4/-q2.3, and my modest tuning -q4. Note that bitrates are not similar between these samples (LC -q4, UC -q4 and MTb2 -q4 is much higher than 128kbps.)

I failed to abx 2 samples because I was not familiar with classical samples and got tired.   

Results:
At a grance, the total score shows that MTb2 was the first. However, it was not so good because the average bitrate of MTb2 is too high (around 160-170kbps). On the other hand, aoTuV has a good cost-performance without bitrate inflation. Preecho problem (e.g., Harpsichord and Mandolins) seems to lower the score (preecho is not his main target of the tuning).

Just as the listening results of guruboolez and [proxima], 1.0.1 official ends up the last place. UC was better than LC.