Can lossy high resolution outperform lossless at standard resolution?

Topic: Can lossy high resolution outperform lossless at standard resolution? (Read 2774 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Can lossy high resolution outperform lossless at standard resolution?

2024-03-30 21:12:36

Hi everyone,
I’m lost and I need some technical opinion.

Context: I’m listening and collecting classical music mostly. I switched to lossless 20 years ago with no regrets. On a large collection including some mono recordings, the average bitrate for FLAC converted CDs is between 580 and 590 kbps. Bitrate is therefore very friendly here and I never felt any need for high bitrate lossy encodings. Like many people I create disposable lossy encodings for my mobile phone.

But in the last decade high resolution PCM (24 bit, from 44100 to 192000 Hz) has become more and more ordinary. I never heard any benefit from the extra resolution but I still decided to favour them. I know it’s irrational, and I don’t even try to understand what’s behind my quest for bigger files (hello Doctor Freud). Furthermore, in the last two years many classical music labels decide to record and sell 192 KHz. And now I must face a serious problem.

Bitrate is strongly inflated at 192 KHz. From less than 600 kbps I have to handle 5000 kbps encodings. That’s clearly very high. I can face a few albums at stratospheric bitrate, but nowadays they multiply very quickly and space is out of control. Even for my ‘storage is cheap’ mantra it’s definitely too much.

So what can I do?
Downsizing 24/96 and 24/192 to 16/44 or maybe 16/48 seems to be the obvious choice. There’s no risk for artefacts or audible noise and bitrate is nice.
But I wonder of it’s the best choice. Couldn’t high resolution lossy encoding be technically superior to standard resolution lossless? For pictures and video, it makes no doubt that high resolution lossy is more enjoyable and technically better than uncompressed video at standard resolution.

Let say I’d like to keep the technical specifications of the HR files: high sampling rate and higher signal-to-noise ratio. What format and what setting should I use to keep the highest quality? What is possibly the best choice for reducing High Resolution bitrate to 800…1200 kbps? How can we measure sound quality at this bitrate?

I currently see these options:
•   Perceptual lossy formats outputting 192.000 Hz: Ogg Vorbis is AFAIK to the only working format/encoder able to reach 192 KHz. I'm pretty sure it's the worst choice (transform and perceptual encoder aren't meant for such high bitrate and resolution)
•   Hybrid lossless formats, like WavPack lossy (I won't use OptimFrog Dual Stream for compatibility reason): amazing tool with many options. I could also reconsider correction files which really make sense with huge downsizing.
•   LossyFLAC / LossyWAV (or LossyTAK for even more efficiency): many presets to play with, FLAC bitstream which is great for compatibility.
•   FSLAC: faster than lossyflac, flac output; but Christian Helmrich doesn't recommend it at high resolution.

Here are the competitors, but I really don’t know how to test them. ABXing 800 to 1000 kbps is not an option. Are there any relevant metrics for audio for formats like WavPack lossy or LossyWav? I know there were some interesting debates in the past, but none for HR files. So well-informed opinions are welcome

NB: I'm currently running a bitrate table in order to check what settings are comparable in size, and to see what can I expect in size at different sampling rates.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #1 – 2024-03-30 21:30:06

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #2 – 2024-03-30 22:09:08

Only way is ABX test, AFAIK, but I am sure you'll soon come to conclusion that 44.1 or 48 kHz is more than enough.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #3 – 2024-03-31 00:24:22

Quote from: guruboolez on 2024-03-30 21:12:36

But in the last decade high resolution PCM (24 bit, from 44100 to 192000 Hz) has become more and more ordinary. I never heard any benefit from the extra resolution but I still decided to favour them.

If ever someone had asked me who honestly may be able to abx highbitrate things i'd answered him @guruboolez

I had several releases at higher samplerates. I do a 24/96 and a 16/44.1 from all that is above and delete even the originals. The 24/96 goes into the backup. The 16/44.1 is on my server for daily playback.
You may wonder but 44.1kHz resampling is transparent to me at least since 20 years and i can't play music loud enough to hear 16bit dither without constructing events. My DAC sounds the same to me in both resolutions.
If i remember right even a James D. (jj) Johnston that knows all the things about human hearing limits finds 64kHz audio is enough for everything even in theory.
There never happened a 64kHz format in public unfortunately so a sane 32kHz filter inside a 88.2kHz or 96kHz file is all youi'll ever need. flac will compress the empty HF area very efficient.
No idea the result is small enough for your needs.

Just my 2 cents

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #4 – 2024-03-31 05:18:03

I'm going to say no, but if you hear a difference you might perceive the lossy file as "better".

If a higher sample rate keeps ultrasonic sounds then the lossy algorithm has to throw-away something else and it might be throwing away something audible. (But I assume the perceptual compression algorithms are smart enough to throw-away ultrasonics.)

MP3 (the lossy format I know the most about) does have more dynamic range than 16-bits. But dynamic range isn't really a problem with CDs or "CD quality", whereas sometimes there are audible MP3 compression artifacts.

Quote

But in the last decade high resolution PCM (24 bit, from 44100 to 192000 Hz) has become more and more ordinary.

And storage has become cheaper.

And you can compress to FLAC which should give you a file around half the size (similar to the compression ratio you get with 16/44.1).

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #5 – 2024-03-31 07:16:32

Hi, as for bit-depth, 24bit + semi-lossy codecs *can* be better than 16bit. At least, I can create such signal artificially.
Attached is a well-known Tom's diner sample attenuated by 96dB.
You can play at normal volume like this:

Code: [Select]

ffplay 24.flac -af volume=96dB
play 24.flac gain 96

16dB one is mostly full of white noise, Suzanne's vocal is barely audible.
On the other hand, 24dB is OK. As can be easily assumed, semi-lossy method cannot compress meaningfully better than simple lossless FLAC (or even worse), since it is too quiet.
Considering real situation, semi-lossy+24dB can efficiently compress loud part and still keep very quiet part.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #6 – 2024-03-31 08:15:24

Off course this is usually non-sense. In normally recorded / normally mastered product, -96dB should be well below noise floor. And it shouldn't be audible anyway with normal volume position... unless you are listening to it with dynamic range compressor enabled.
Logically, nothing can stop somebody from exploiting rich dynamic range provided by hi-res, and nothing can stop us from using dynamic range compressor.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #7 – 2024-03-31 09:24:52

Quote from: Gabriel Schwartz on 2024-03-30 21:30:06

No

Wow, so much to read

I could rephrase it differently: what would you choose at 300 kbps: a lossy encoding at 44.000 Hz (AAC, Opus) or a lossless encoding (FLAC) at 22.000 Hz or even 16.000 Hz which has no artifact but a muffled sound? I think I know the answer...
At a given bitrate, a lossy stream can produce (much) better quality than a lossless one. But in the case of lossless RedBook vs lossy HR we have to face transparency at both side. HR files have for example a better SNR but the lossy tools probably reduce the SNR as well. The answer is therefore much more theoretical. Hence my questioning.

Quote from: itisljar on 2024-03-30 22:09:08

Only way is ABX test, AFAIK, but I am sure you'll soon come to conclusion that 44.1 or 48 kHz is more than enough.

For sure! I came to the conclusion that 144 kbps AAC is already very close to transparency. The ABX tells me that 200 kbps lossy with any modern format is more than enough for my rational needs.
In the past, here on HA.org which is ruled by TOS#8, people were allowed to use some datasheet or graphs for arguing on debate that are beyond hearing ability: DSD vs PCM, or LossyWav settings, hybrid format vs another hybrid format. It's what I'm looking for: is there any way to measure (or at least for trying) sonic properties (dynamic range, noise level, etc...) of ~800 kbps encodings which don't rest on perceptual techniques?

Quote from: Wombat on 2024-03-31 00:24:22

I had several releases at higher samplerates. I do a 24/96 and a 16/44.1 from all that is above and delete even the originals. The 24/96 goes into the backup. The 16/44.1 is on my server for daily playback.
Just my 2 cents

It's an interesting advice. Put a cap at 24/96 is indeed a way to limit the growing size of modern recordings. I'll nevertheless try to reduce further the bitrate (at least for some albums I don't like that much)

Quote from: DVDdoug on 2024-03-31 05:18:03

And storage has become cheaper. And you can compress to FLAC which should give you a file around half the size (similar to the compression ratio you get with 16/44.1).

Oups, I wasn't clear enough. I'm already using FLAC myself. Uncompressed stereo 24/192 kbps recordings = 9200 kbps. On average I already reach ~5000 kbps with FLAC (bitrate varies from 3000 to 7000 kbps). For 24/96 kbps, bitrate is around 2500 kbps with FLAC on my side.

Quote from: nu774 on 2024-03-31 07:16:32

Hi, as for bit-depth, 24bit + semi-lossy codecs *can* be better than 16bit. At least, I can create such signal artificially.
Attached is a well-known Tom's diner sample attenuated by 96dB.

Hi, and thank you for you amazing work with LossyWAV. I appreciate your contribution here.
When you develop lossyWAV, do you have a meaningful tool or methodology to measure the effect on sound "quality"?

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #8 – 2024-03-31 09:39:28

Quote from: guruboolez on 2024-03-31 09:24:52

Quote from: nu774 on 2024-03-31 07:16:32
Hi, as for bit-depth, 24bit + semi-lossy codecs *can* be better than 16bit. At least, I can create such signal artificially.
Attached is a well-known Tom's diner sample attenuated by 96dB.
Hi, and thank you for you amazing work with LossyWAV. I appreciate your contribution here.
When you develop lossyWAV, do you have a meaningful tool or methodology to measure the effect on sound "quality"?

I think you may be slightly confused, the author of lossyWAV is Nick.C

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #9 – 2024-03-31 09:44:13

Sorry for my mistake

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #10 – 2024-03-31 09:50:54

Quote from: guruboolez on 2024-03-30 21:12:36

Downsizing 24/96 and 24/192 to 16/44 or maybe 16/48 seems to be the obvious choice. There’s no risk for artefacts or audible noise and bitrate is nice.
But I wonder of it’s the best choice. Couldn’t high resolution lossy encoding be technically superior to standard resolution lossless? For pictures and video, it makes no doubt that high resolution lossy is more enjoyable and technically better than uncompressed video at standard resolution.

Lossy encoders like AAC, OPUS, OGG Vorbis are optimized for the human hearing range up to 20kHz. I assume there is no valid model for frequencies >20kHz in Perceptual lossy formats.
I've checked in the past a lot of recordinga >96kHz samplerate. In general it contained just noise above sound frequencies 48kHz.
In general there is no speaker driver build and optimised for frequencies > 24kHz. For shure no one for 48kHz and up. (I know there are some out there, but not in mass build products)
Intermodulation artefacts in your Amplifier, can harm the fidelity in the hearing range.

Best option is to resample to 24bit/96kHz or to save more space 24/48kHz.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #11 – 2024-03-31 12:54:12

Just to come back to the need of bandwith in the name of science.
Here James D. (jj) Johnston says exactly this "...filter, that one starting to cut off at 25kHz and finishing at 32kHz. Shorter, no possible interference in the time domain based on current understandings of cochlear dynamics." He even suggests the formula.
Combine that with resampling to 18bit, slightly noise-shaped and flac compresses the empty lower bits and the non existing HF in a 24/96kHz file just fine. Call it flac for humanoids.
This may be the honest way for what MQA wanted to be.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #12 – 2024-03-31 14:17:55

Storage is still cheap, there is no reason to use SSD for the off-site collection

A possible solution first - but maybe @bryant will weigh in on the usefulness at "transparent" levels and possibly shoot it down:
* On your big spinning off-site drive with all the junk on it: WavPack hybrid lossless
* On your computer's SSD, where storage isn't cheap: Only the .wv files.

A possibly interesting thread:
@Nick.C 's thread on TransPCM with float16 and later float24 here: https://hydrogenaud.io/index.php/topic,90770.0.html . Float sounds ideal for classical music with lots of more quiet parts - unfortunately, there isn't much support.
@bryant points out that it isn't unlike WavPack lossy.

On a general note, "everything here involves lossy operations" of course, and the question is whether one can do damage - could naive downsampling be prone to clipping? In the very least, decimating to integer format must be done right.
Not that it should be a problem - if you go 6 dB down, you might just use an extra bit at the bottom as long as you stay clear of ALAC/Monkey's which don't utilize wasted bits.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #13 – 2024-03-31 14:40:32

The first world problem, huh? Okay, let's put this question into perspective, because snake oil is infinite, to paraphrase the quote attributed to Albert Einstein, and 192 kHz is no longer the limit of what you can encounter. Here is a modern music cover that informs the source comes with a sampling rate of 352.8 kHz. And below that we see a piece of not so über-expensive audio equipment that claims to handle even 768 kHz.

Since Vorbis cannot climb that high, it's no longer an option. Raise the fist and draw some air: WavPack, WavPack, WavPack! Because it supports sampling rates up to 1 GHz, right? Let's take some 4 minutes 192 kHz 24-bit song, process it with SoX resampler DSP if necessary, and witness how WavPack 5.70 -b4x3 hybrid (without correction files) scales.

Code: [Select]

        Bytes    BPS          Filename  
 ------------- ------ ----------------- 
   11 046 164    373    44100.lossy.wv  
   12 001 612    406    48000.lossy.wv  
   21 792 564    737    88200.lossy.wv  
   23 802 134    805    96000.lossy.wv  
   43 732 116   1479   176400.lossy.wv  
   47 616 420   1611   192000.lossy.wv  
   88 388 268   2990   352800.lossy.wv  
  204 902 192   6933   768000.lossy.wv

The output grows, no surprise there. So it's just a matter of time before lossy files that preserve that Burj Khalifa sampling rate and Kola Borehole bit depth of the source force you to worry about free space again. That is, what you are experiencing now with bloated lossless files will happen later with bloated lossy files. As Karl Marx put it, “first as a tragedy, then as a farce”. The good news is you have a hunch on how to proceed.

Quote from: guruboolez on 2024-03-30 21:12:36

Downsizing 24/96 and 24/192 to 16/44 or maybe 16/48 seems to be the obvious choice.

This. And you'll still be able to hear the owl flying by, if, of course, the microphone captured it in the first place.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #14 – 2024-03-31 15:13:49

Quote from: Porcus on 2024-03-31 14:17:55

A possibly interesting thread:
@Nick.C 's thread on TransPCM with float16 and later float24 here: https://hydrogenaud.io/index.php/topic,90770.0.html . Float sounds ideal for classical music with lots of more quiet parts - unfortunately, there isn't much support.
@bryant points out that it isn't unlike WavPack lossy.

If using TransPCM to produce FP16 PCM please note that the output is full scale, i.e. ±65504.0, not ±1.0.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #15 – 2024-03-31 15:35:03

Quote from: Kraeved on 2024-03-31 14:40:32

192 kHz is no longer the limit of what you can encounter. Here is a modern music cover that informs the source comes with a sampling rate of 352.8 kHz.

Format introduced nearly twenty years ago. Those files didn't hit the retail until bandwidth got a bit more manageable, but this kind of resolution was part of the 2L label's free-for-download high resolution testbench that was discontinued some time ago because ... well because too old news already I guess.
352.8/24 was/is named "DXD" to suggest a connection to DSD, likely to fool users into thinking that DSD is about that good.

Here you got some PR stunt that outdoes DXD - and looks about as stooopid as this:

In https://hydrogenaud.io/index.php/topic,93853.msg1006575.html#msg1006575 , guruboolez points out it was originally recorded at 352.8/24 , then mixed in the analog domain and then re-digitized at 768 kHz for ... reasons. Getting you the tape hiss of the Studer.
Compression figures here: https://hydrogenaud.io/index.php/topic,122179.msg1011846.html#msg1011846

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #16 – 2024-03-31 15:59:59

wavpack around 6 bits per sample (-b6x4c) . Bitrate will scale to 2000-2500k for 192khz . For sampling rate > 48khz & 6bps , Bryant says all the noise is shifted up ( via 1st order noise shaping) into bat hearing range. For around 1500k ; -b4x4c ( scales to 352k for 44-16) should be a nice option to try. Maybe down to 3.5bps. The noise will
still be shaped exactly the same ( pushed into the HF ) as its default for above 48khz.

I have some bandcamp 24bit stuff. it can save a lot the lossy file remains same size as 16 bit.

https://www.wavpack.com/wavpack_doc.html#noncd

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #17 – 2024-03-31 16:09:59

I also have a bit of this dilemma; bandcamp etc . I also created flac, wav, and WV presets in F2k to output 16bit-44.1 as well.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #18 – 2024-03-31 16:25:49

Quote from: Porcus on 2024-03-31 15:35:03

Here you got some PR stunt that outdoes DXD - and looks about as stooopid as this:

That spectrogram looks similar to what we get from 1bit DSD in SACD.
Off course wasteful bullshit, but what bugs me is that SACD can be actually mastered better than CD, not tortured by insanely high compression due to the loudness war.

Quote from: shadowking on 2024-03-31 16:09:59

I also have a bit of this dilemma; bandcamp etc .

Exactly. Since bandcamp shows only formats, I usually notice it much later.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #19 – 2024-03-31 16:39:10

Quote from: Porcus on 2024-03-31 15:35:03

352.8/24 was/is named "DXD" to suggest a connection to DSD

There is. As DSD is not an editable format, DXD was developed for the Merging Pyramix workstation and introduced together with their Sphynx 2, AD/DA converter in 2004. This combination meant that it was possible to record and edit directly in DXD as it is plain PCM.
The final product is converted to DSD and will sound of course superior to PCM.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #20 – 2024-03-31 17:12:27

Oh sure it has a connection in name even if no connection in format, that was my point. Capturing pulse-width modulation from a sigma-delta ADC before making integer PCM out of it would ~~save a step in the process~~ leave a step in the process for later. Meaning, if conversion to integer-PCM introduces clipping, you have an earlier file that hasn't.

Then of course it turns out, it isn't possible to edit it, and then they invented the "DXD" phrase that suggests it is "editable" DSD and not plain integer PCM. They should have used float, of course.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #21 – 2024-03-31 18:00:20

Quote from: Kraeved on 2024-03-31 14:40:32

Since Vorbis cannot climb that high, it's no longer an option. Raise the fist and draw some air: WavPack, WavPack, WavPack! Because it supports sampling rates up to 1 GHz, right? Let's take some 4 minutes 192 kHz 24-bit song, process it with SoX resampler DSP if necessary, and witness how WavPack 5.70 -b4x3 hybrid (without correction files) scales.

I only accept recordings with a sample rate that saturates my 4 GHz computer CPU.

Nah. I remember as a teenager being annoyed by high frequencies that apparently adults couldn't hear. Seeing the reaction dogs have to dog whistles at ~35 kHz, I think recordings with information above 20 kHz is not a feature. It's a bug. I would go out of my way to obtain speakers that DO NOT reproduce beyond 20-24 kHz.

I'm in luck, since the majority of consumer products DO NOT reproduce beyond 20-24 kHz anyway, regardless of what you feed them.

Maybe if people had their ears bleed more often with such high-pitched tones, they would think twice about advertising it as a positive thing.

( Stargate SG-1 Season 2 Episode 19, main characters getting headaches from long exposure to high-pitched sounds )

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #22 – 2024-03-31 20:39:51

Quote from: .halverhahn on 2024-03-31 09:50:54

I've checked in the past a lot of recordings >96kHz samplerate. In general it contained just noise above sound frequencies 48kHz.
(...)
Best option is to resample to 24bit/96kHz or to save more space 24/48kHz.

I have selected 35 albums from 35 different labels, all sampled at 192 Khz in order to build a bitrate table and make eventually some tests.
You can see in the picture at the bottom a frequency graph of 30 seconds of each album merged into one file. The noise level differs a lot from an album to another. Some recordings are noise free, but many have dither noise. Many recordings also have high-pitched frequency bands at different level. Is it electronic noise generated by some devices, like 15.625 kHz TV signal?
The fifth recording from the left has an insane amount of HF noise and is the less compressible signal (7000 kbps in FLAC). It comes from a small japanese label.
But I agree that none of these 35 albums present any useful information beyond 48000 Hz.

Quote from: Wombat on 2024-03-31 12:54:12

Combine that with resampling to 18bit, slightly noise-shaped and flac compresses the empty lower bits and the non existing HF in a 24/96kHz file just fine. Call it flac for humanoids.
This may be the honest way for what MQA wanted to be.

That's a very interesting solution, thanks! Bitrate should stay below 1000 kbps, format remain highly compatible, and the remaining data should logically offer a superior quality to RedBook specification.

Quote from: Porcus on 2024-03-31 14:17:55

A possibly interesting thread:
@Nick.C 's thread on TransPCM with float16 and later float24 here: https://hydrogenaud.io/index.php/topic,90770.0.html . Float sounds ideal for classical music with lots of more quiet parts - unfortunately, there isn't much support.
@bryant points out that it isn't unlike WavPack lossy.

Quote from: Nick.C on 2024-03-31 15:13:49

If using TransPCM to produce FP16 PCM please note that the output is full scale, i.e. ±65504.0, not ±1.0.

I never heard of TransPCM before. I'll also take a look. Thank you to both of you!

@shadowking > thanks for your advice. Your long experience with WavPack lossy is very useful and to be fair I was waiting for your contribution

But -b6 seems really high. I was expecting WavPack lossy to be more efficient with HR files (more room to handle noise). Am I wrong?

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #23 – 2024-03-31 20:52:07

Quote from: guruboolez on 2024-03-31 20:39:51

I never heard of TransPCM before. I'll also take a look. Thank you to both of you!

I should also clarify that it does not change sampling rate, just the "container" in which each sample is contained, e.g. 24-bit integer > 16-bit float.

Re: Can lossy high resolution outperform lossless at standard resolution?

Reply #24 – 2024-03-31 22:04:01

Quote from: guruboolez on 2024-03-31 20:39:51

The fifth recording from the left has an insane amount of HF noise and is the less compressible signal (7000 kbps in FLAC).
[...]
But I agree that none of these 35 albums present any useful information beyond 48000 Hz.

The fifth recording looks like a straigth DSD64 to PCM converting without any filtering. Looking at the freq. plot, a 48kHz sampling would be sufficient for track 5.

Quote from: guruboolez on 2024-03-31 20:39:51

Quote from: Wombat on 2024-03-31 12:54:12
Combine that with resampling to 18bit, slightly noise-shaped and flac compresses the empty lower bits and the non existing HF in a 24/96kHz file just fine. Call it flac for humanoids.
This may be the honest way for what MQA wanted to be.
That's a very interesting solution, thanks! Bitrate should stay below 1000 kbps, format remain highly compatible, and the remaining data should logically offer a superior quality to RedBook specification.

Foobar 2000 and the mda dither DSP component for foobar2000 is quite handy to do the job.
It will reduce bitdepth to e.g. 20bit, apply noise shaped dither and save it into a 24bit FLAC (lower bits are empty).

Notice