Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Why 256kbps vorbis is better than 512kbps opus? (Read 2548 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Why 256kbps vorbis is better than 512kbps opus?

I'v made a python script to compare different encoding methods from lossless wave file.

Name                     PNSR L   PNSR R   Similarity L  Similarity R
qaac-320kbps.wav         45.762   45.763   0.99982       0.99984     
mp3-320kbps.wav          44.663   44.844   0.99977       0.99980
vorbis-320kbps.wav       43.631   43.535   0.99973       0.99974
mp3-245kbps.wav          41.480   41.860   0.99953       0.99960
qaac-256kbps.wav         41.363   41.378   0.99952       0.99955
vorbis-256kbps.wav       39.826   39.782   0.99932       0.99936
opus-512kbps.wav         37.079   36.764   0.99870       0.99870    <- what's wrong with this?
opus-320kbps.wav         35.666   35.377   0.99820       0.99821
opus-256kbps.wav         34.658   34.339   0.99773       0.99773
qaac-128kbps.wav         32.440   32.495   0.99620       0.99652
mp3-130kbps.wav          31.219   31.359   0.99497       0.99548
vorbis-128kbps.wav       30.675   30.366   0.99428       0.99430
opus-128kbps.wav         30.363   30.185   0.99386       0.99407
48-44-48-ssrc.wav        23.114   23.203   0.96725       0.97024
48-44-48-retroarch.wav   22.372   22.439   0.96113       0.96450

My test step:
1. Prepare a 48khz wav file.
2. Convert it to different encodings though foobar with all the latest encoders.
3. Convert the files from step 2 to .wav files.
4. Use the python script to compare the PNSR and cosine similarity. (the script is in the attachment)

Result:
1. Apple AAC(QAAC) and MP3 are really good.
2. Any encoder with 128kbps can have above 0.99 cosine similarity with the reference.
3. This is just a test of one 48khz file, you can have your own test. (the script is in the attachment)

Bonus:
48-44-48-ssrc.wav is generated in foobar from: source -> 44.1khz by ssrc -> 48khz by ssrc ->48-44-48-ssrc.wav
48-44-48-retroarch.wav is generated in foobar from: source -> 44.1khz by RetroArch Highest -> 48khz by RetroArch Highest->48-44-48-retroarch.wav

How to use the script:
1. Install python and numpy, soundfile packages.
2. Put the test.py script in a folder and put a GroundTruth.wav in the same folder.
3. Create a folder named wav in the same folder.
4. Put all the converted wav files in the wav folder.
5. Use: python test.py to run the script.

Question:
Why 256k vorbis is better than 512k opus?

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #1 –
Quote from: qdwang on
PNSR

I think you meant PSNR.

Other than that, this measurement is pretty irrelevant when it comes to (perceived) sound quality, so subsuming the result of this measurement under a claim that something is "better" than something else is rather misguided and misleading.

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #2 –
Yep, in case of psychoacoustic-based lossy compression, PSNR isn't a reliable indicator. Neither are spectrograms. Nor listening to whats left after extracting the encode from original.
Blind listening test is the way to go.

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #3 –
What this test shows is that MP3, LC-AAC and Vorbis use similar technologies, while opus uses a different one.

Also, the fact that a high quality resampling gives you a worse value than any of the other encodings even for the lower quality settings should be a reason enough to discard this as a proper way to evaluate lossy codecs.

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #4 –
Quote from: doccolinni on
Quote from: qdwang on
PNSR

I think you meant PSNR.

Other than that, this measurement is pretty irrelevant when it comes to (perceived) sound quality, so subsuming the result of this measurement under a claim that something is "better" than something else is rather misguided and misleading.

Thanks for correcting me. Is there a more scientific way to detect which one is closer to the original one rather than blind listening?

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #5 –
Quote from: [JAZ] on
What this test shows is that MP3, LC-AAC and Vorbis use similar technologies, while opus uses a different one.

Also, the fact that a high quality resampling gives you a worse value than any of the other encodings even for the lower quality settings should be a reason enough to discard this as a proper way to evaluate lossy codecs.

Thanks for the reply. But is there a more scientific method to determine which one is closer to the original, instead of relying on blind listening?

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #6 –
Blind ABX listening tests are very scientific. They're the best way to determine which one sounds closer to the original.

PSNR and cosine similarity can be useful metrics, but they don't tell you which one sounds closer to the original.

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #7 –
Quote from: Octocontrabass on
Blind ABX listening tests are very scientific. They're the best way to determine which one sounds closer to the original.

PSNR and cosine similarity can be useful metrics, but they don't tell you which one sounds closer to the original.

Thank you, now i get it.

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #8 –
Quote from: qdwang on
opus-512kbps.wav         37.079   36.764   0.99870       0.99870    <- what's wrong with this?
This is not necessarily explaining your numbers, but....
Encoding any lossy codec @ 512 kbps? A lot! ^^
For example that i have some flac (!) files that go as low as 250 kbps. Another thing would be that beyond a certain point lossy codecs are transparent, so there is no need for further bandwidth.
And so, with digital, computer was put into place, and all the IT that came with it.

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #9 –
Quote from: Octocontrabass on
Blind ABX listening tests are very scientific. They're the best way to determine which one sounds closer to the original.
If I'm not mistaken, ABX can only tell you if the lossy version is transparent or not. If you want to score multiple lossy versions to see which one sounds the closest to the original, you need to perform an ABC/HR test, not ABX.
 

Re: Why 256kbps vorbis is better than 512kbps opus?

Reply #10 –
You're right, I got my blind listening tests mixed up.