Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Can SoX handle float file with volume over 0dBFS? (Read 11761 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Can SoX handle float file with volume over 0dBFS?

I have such files and foobar shows 7.079733 in replaygain scan (track peak) but when I use SoX's stats command I got the following

Code: [Select]
Input File     : 'D:\bennet\@\sox\float.wav'
Channels       : 2
Sample Rate    : 44100
Precision      : 24-bit
Duration       : 00:00:26.43 = 1165375 samples = 1981.93 CDDA sectors
File Size      : 9.32M
Bit Rate       : 2.82M
Sample Encoding: 32-bit Floating Point PCM

In:100%  00:00:26.43 [00:00:00.00] Out:1.17M [!=====|=====!]        Clip:926k
             Overall     Left      Right
DC offset   0.000284  0.000284  0.000283
Min level  -1.000000 -1.000000 -1.000000
Max level   1.000000  1.000000  1.000000
Pk lev dB       0.00      0.00      0.00
RMS lev dB     -3.61     -3.61     -3.61
RMS Pk dB      -0.27     -0.27     -0.27
RMS Tr dB      -1.#J     -1.#J     -1.#J
Crest factor       -      1.51      1.51
Flat factor    37.26     37.26     37.26
Pk count        463k      463k      463k
Bit-depth      32/32     32/32     32/32
Num samples    1.17M
Length s      26.426
Scale max   1.000000
Window s       0.050
Done.


I tried SoX's −G and the gain and norm commands to process the files but clipping distortion persists. In foobar I can use replaygain>preamp to reduce volume without distortion. Is this SoX's limitation? Thanks!

Can SoX handle float file with volume over 0dBFS?

Reply #1
I see the file is stored in floating point (as suggested by the name, but confirmed by Sox with "Sample Encoding: 32-bit Floating Point PCM" (and indirectly by the Bitrate, which is double of stereo 16bit 44Khz).
But said that, Sox is not operating in 32bits floating point, but in 24bit integer as indicated by "Precision      : 24-bit".

That's why min and max level are clipped to 1, which is the max value that can be stored in integer (because it represents full scale).

I don't know/use sox, but in case it can use the gain when doing stats, and you say that the file has a peak of 7, you should be applying a gain of 0.14.
I don't really know if it might be a bug, an implementation limitation, or an erroneous use of commandline parameters (maybe you need to specify the output format).

Can SoX handle float file with volume over 0dBFS?

Reply #2
I guess SoX really unable to deal with float files above 0dBFS. Just tried TAudioConverter > Filters > SoX > Guard against clipping, but files still clipped even when writing to 32-bit float format.

Can SoX handle float file with volume over 0dBFS?

Reply #3
Out of curiosity, i've downloaded 14.4.1 and played with it a bit.

The sad thing is that you are correct.



The instruction
Code: [Select]
>sox -S testfloat.wav -t wav -e floating-point tmp.wav
clips the output just like you say, even when the input file is in floating point, and i am asking the output to be floating point (which is is, but clipped).
Adding -G didn't change anything (i guess that's for when it applies multiple effects).

Finally, i tried the following:
Code: [Select]
>sox -S testfloat.wav -t sox -e floating-point tmp.wav
And that generates a 32bit *integer* file.

So it's like sox works internally in 32bit integer, although that's sort of incompatible with having a "clip guard".


Not even a gain operation on the input file is free of that:
Code: [Select]
>sox -v 0.1 -S testfloat.wav -t wav -e floating-point tmp.wav
(i.e. reduce the volume at 10% on the source)
Reduces the volume, but after clipping the input.
(again, same result with or without -G)

I tried explicitely a float format, but still the same result:
Code: [Select]
>sox -S -t f32 -e floating-point -b 32 --endian little -c 2 -r 44100 testfloat-raw.wav -t f32 -e floating-point -b 32 --endian little -c 2
-r 44100 tmp.wav

Can SoX handle float file with volume over 0dBFS?

Reply #4
Internally sox operates in int32. Therefore, all input formats are converted to sox_sample_t first.

http://sourceforge.net/p/sox/code/ci/maste.../src/sox.h#l894

The variable clip will contain number of samples outside of the allowed range -1, +1 in the input stream.

Therefore, it is not possible to use sox for IMO incorrect files with float samples larger than 1 without clipping.

Can SoX handle float file with volume over 0dBFS?

Reply #5
Ok, after reading a bit more the manual and that, the options that sox offer to prevent clipping are in the context of effects/switches (so, the result). In any case, above full scale is always clipped, and the options either advise about the clipping, or apply a gain when it sees it will clip.

But that means that, if I wanted to decode an MP3 file with sox (and apply whatever effect i wanted), the origin will probably be already clipped if it is not mp3gained. (I hope this example is more real than the "float with above full scale values" situation).

Feels strange... it's a bit misleading to have options to operate in float, but not have the benefits of what float means.

Re: Can SoX handle float file with volume over 0dBFS?

Reply #6
New day, new discoveries on Hydrogen. Now I have found out that mighty SoX does not work internally with floating point. One might think this feature should be prioritized as soon as possible, but the latest version 14.4.2 has been frozen for 8 years already. “Life will never be the same again”.
• Join our efforts to make Helix MP3 encoder great again
• Opus complexity & qAAC dependence on Apple is an aberration from Vorbis & Musepack breakthroughs
• Let's pray that D. Bryant improve WavPack hybrid, C. Helmrich update FSLAC, M. van Beurden teach FLAC to handle non-audio data

Re: Can SoX handle float file with volume over 0dBFS?

Reply #7
I mention such stuff several times, but people keep using sox for some reason I will never understand.
Please remove my account from this forum.

Re: Can SoX handle float file with volume over 0dBFS?

Reply #8
@mycroft, it's not that easy to replace SoX with, say, FFMPEG. For example, you want to convince a person that he doesn't need to encode frequencies above, say, 18 kHz, because he most likely doesn't hear them. To achieve this, we take this person’s favourite composition and then cut off below the mentioned threshold as follows:

sox in.wav out.wav sinc 18k-0k

Now, what can FFMPEG offer in this case? I tried, tried again, but no luck so far.
• Join our efforts to make Helix MP3 encoder great again
• Opus complexity & qAAC dependence on Apple is an aberration from Vorbis & Musepack breakthroughs
• Let's pray that D. Bryant improve WavPack hybrid, C. Helmrich update FSLAC, M. van Beurden teach FLAC to handle non-audio data

Re: Can SoX handle float file with volume over 0dBFS?

Reply #9
@mycroft, it's not that easy to replace SoX with, say, FFMPEG. For example, you want to convince a person that he doesn't need to encode frequencies above, say, 18 kHz, because he most likely doesn't hear them. To achieve this, we take this person’s favourite composition and then cut off below the mentioned threshold as follows:

sox in.wav out.wav sinc 18k-0k

Now, what can FFMPEG offer in this case? I tried, tried again, but no luck so far.

Huh!

Just quickly searching the manual page, I arrived at this:

Code: [Select]
ffmpeg -i in.wav -af firequalizer=gain='if(gt(f\,18000)\, 0\, -INF)' out.wav

Re: Can SoX handle float file with volume over 0dBFS?

Reply #10
@2012, bravo, it indeed works.

However, I believe many would agree that this command looks so unintuitive (FIR Equalization? Some gain? Minus infinity?) and complicated (all those quotes and especially slashes) that the only way to use it is to save it somewhere and paste it in as needed. I could even add it to my signature, but I don't want to scare people. Who knows, maybe there is an easier way, since the manual mentions commands like bandpass and bandreject, and also maybe highpass filter has some option I missed to make it really sharp.

Now, let's compare SoX sinc and FFMPEG firequalizer.
The cutoff is sharper and the noise is less noticeable in the latter case.

Spoiler (click to show/hide)
• Join our efforts to make Helix MP3 encoder great again
• Opus complexity & qAAC dependence on Apple is an aberration from Vorbis & Musepack breakthroughs
• Let's pray that D. Bryant improve WavPack hybrid, C. Helmrich update FSLAC, M. van Beurden teach FLAC to handle non-audio data

Re: Can SoX handle float file with volume over 0dBFS?

Reply #11
Now, let's compare SoX sinc and FFMPEG firequalizer.
The cutoff is sharper and the noise is less noticeable in the latter case.
By default SoX applies dither on 16-bit files. That's probably what you are seeing.

The transition band in SoX sinc is 5% by default but can be changed with "-t". "-t 200" gives something similar to ffmpeg firequalizer.

And as per your first SO link, fireqaulizer still doesn't null in the passband with the original (maybe it does with some additional settings, I don't know).

Re: Can SoX handle float file with volume over 0dBFS?

Reply #12
@danadam, I've managed to make SoX cut as sharp as FFMPEG and disable dithering (get rid of extra noise) as follows

sox in.wav out.wav -D sinc 18k-0k -n 32767
• Join our efforts to make Helix MP3 encoder great again
• Opus complexity & qAAC dependence on Apple is an aberration from Vorbis & Musepack breakthroughs
• Let's pray that D. Bryant improve WavPack hybrid, C. Helmrich update FSLAC, M. van Beurden teach FLAC to handle non-audio data

Re: Can SoX handle float file with volume over 0dBFS?

Reply #13
FFmpeg have sinc filter, if you ever bothered to read documentation.
sinc ffmpeg implementation is more correct that sox one.
Usage of sinc filter with ffmpeg is left as exercise to non-ignorant readers.
Please remove my account from this forum.

Re: Can SoX handle float file with volume over 0dBFS?

Reply #14
@mycroft, I read a lot, and not just technical documentation, so I can compare and argue that SoX is easier to use, even without fully understanding the underlying concepts, such as some kaiser not-Wilhelm windows and some non-kitchen taps. Its capabilities are well documented with examples. As for FFMPEG, this picture is worth a thousand words.



And to change situation for the better, we need to fill in the gaps, thereby helping others to grow, rather than putting them down by implying ignorance. In particular, Google does not provide enough relevant results for the keywords ffmpeg and sinc. Personally, by trial and error, I found that the following command works in a way, frequencies below 18 kHz are definitely cut, but the rest is puzzling: the noise of the original is missing and the peaks look different, which neither SoX sinc nor FFMPEG firequalizer did.

ffmpeg -i in.wav -f lavfi -i sinc=hp=18000:hptaps=32767 -lavfi afir out.wav


• Join our efforts to make Helix MP3 encoder great again
• Opus complexity & qAAC dependence on Apple is an aberration from Vorbis & Musepack breakthroughs
• Let's pray that D. Bryant improve WavPack hybrid, C. Helmrich update FSLAC, M. van Beurden teach FLAC to handle non-audio data

Re: Can SoX handle float file with volume over 0dBFS?

Reply #15
Google does not provide enough relevant results for the keywords ffmpeg and sinc.
Looks like HA is not very relevant for google: HowTo .dsf --> .flac - Reply #14

frequencies below 18 kHz are definitely cut, but the rest is puzzling: the noise of the original is missing and the peaks look different, which neither SoX sinc nor FFMPEG firequalizer did.

ffmpeg -i in.wav -f lavfi -i sinc=hp=18000:hptaps=32767 -lavfi afir out.wav
Apparently with this invocation the signal level is reduced by ~6 dB, so the noise disappeared below the scale:
Code: [Select]
]$ sox -r44.1k -n -b16 sin20k.wav synth 30 sin 20k norm -.0004 fade h .5 0 .5

]$ ffmpeg -hide_banner -loglevel warning -y -i sin20k.wav -f lavfi -i sinc=hp=18000:hptaps=32767 -lavfi afir out.wav

]$ soxpeaks.sh sin20k.wav out.wav
Sample peak,          RMS
 -0.00 dBFS,   -3.08 dBFS - sin20k.wav
 -5.88 dBFS,   -8.97 dBFS - out.wav
Maybe there are some settings to prevent that, I don't know.

Generally my attitude to ffmpeg is best summarized at 1:09:

- I haven't found the command yet but it does support it... it does support it.

https://www.youtube.com/watch?v=9kaIXkImCAM

All in all the the lack of float support in SoX was never a problem for me and I agree that its interface is miles better than that of ffmpeg (IMO), so that's what I use. For the rare occasions when I need a lossy file or float wav loaded without clipping I can switch to ffmpeg.

Re: Can SoX handle float file with volume over 0dBFS?

Reply #16
I dislike users that just take first available shortcut someone else made already.
First available shortcut is not documentation.
Documentation are documented options for each filter.
I like to reinvent stuff to be better, faster and perfect in both implementation and design perspective not to do hacks or absurd limitations like SoX does.
For the afir filter to force not normalization of IR audio (which is by default set to very conservative values to not cause clippings in convolution output) in FFmpeg use afir=gtype=none and in Librempeg (better project than FFmpeg in any measurable way) use afir=irnorm=-1.

If you want even better quality there is option to toggle full double floating-point precision in convolution, I may also add to sinc filter to generate real double floating-point output for even better precision when filtering.
Please remove my account from this forum.



Re: Can SoX handle float file with volume over 0dBFS?

Reply #19
New day, new discoveries on Hydrogen. Now I have found out that mighty SoX does not work internally with floating point. One might think this feature should be prioritized as soon as possible, but the latest version 14.4.2 has been frozen for 8 years already. “Life will never be the same again”.
Here's the thread from 2023 on SoX mailing list about that: https://sourceforge.net/p/sox/mailman/sox-users/thread/20230522070824.6g2tkoksss72ay5t%40fastmail.com/

Re: Can SoX handle float file with volume over 0dBFS?

Reply #20
@mycroft

Using sinc+afir seems to introduce a delay, but keeps the number of samples the same, which means the end is truncated! This still happens in latest librempeg.

Is that correct? And if yes, is it a bug/limitation/me doing something wrong?

Also, it would be great if the delay was removed from the start and end of firequalizer output.

Re: Can SoX handle float file with volume over 0dBFS?

Reply #21
It makes sense to talk about single IR delay value only for linear phase IRs. Otherwise delay is not constant and could be average or anything between min and max delay. Also latest librempeg automatically adjust PTS/timestamps when using linear phase IRs, so for manipulating input files (instead of actual real-time audio stream) just use after afir filter, atrim=start=0 if stream starts from 0, this will obviously not work if file/stream PTS/timestamps not starts from 0 values but I will look what is the best generic way to handle that. As output of convolution is number of samples of source + number of samples in IR minus one sample, it should not be trimmed, this is especially relevant/important for linear phase IR. Will also look how best to handle this. Current workaround is to use apad filter for first input source audio stream.
Please remove my account from this forum.

Re: Can SoX handle float file with volume over 0dBFS?

Reply #22
@mycroft 

Thanks for the tips. `atrim=start=0` works as you described.

An option to auto-pad then end-trim `pad_dur-delay` would be ideal, I think.