Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 32-bit floating-point (WAV) files "in the wild" (Read 7540 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

32-bit floating-point (WAV) files "in the wild"

There are a few of these out on the 'net by now. I can only speculate that more artists use software that work with such files, and more often than before an artist will upload a file without having to worry whether to save bandwidth by going mp3.

But why? Of course there will always be someone who on purpose will upload the biggest files they can, but I know that not all of my files were intended to be uploaded in that format ...
Is it so that these DAWs do not even tell you that these are not your "usual" WAV files? (What does Audition do? I'm too lazy to download a trial version just for this.)

Just curious.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #1
In my little world of Linux/Unix and Windows command line, I use the utility sox (the command name is actually soxi), and that is how I normally determine what kind of data/sample rate in a wav/mp3/flac/etc file.   The utiiliies that I write also automatically display the data type/rate.   My own 'normal' data type is 32bit float (or 24bit signed int for flac), even though I normally produce 16bit to send to others.  So, the best bet that costs NO money is to use the command line soxi.   Also, I happen to know that Audacity (free GUI audio software) displays the data type also ( i use Audacity once in a while.)   A wonderful (almost lossless) tradeoff is to use 24bit signed flac, which ends up being not far different from a 16bit wav file in size.

John

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #2
A very horrible thing with float 32 file is they can be encoded in different "normalized" fashions. If the decoder misinterpret the format, the result will be horrible full scale square wave-ish like crap which may destroy your equipment. I experienced this nightmare several times, sometimes in foobar2000, sometimes in Audition.

Of course, in DAWs if people find the decoded waveform looks like crap, they won't hit the play button, but that's not the case with typical audio players.

https://forum.cockos.com/showthread.php?t=85641

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #3
A very horrible thing with float 32 file is they can be encoded in different "normalized" fashions. If the decoder misinterpret the format, the result will be horrible full scale square wave-ish like crap which may destroy your equipment. I experienced this nightmare several times, sometimes in foobar2000, sometimes in Audition.

Of course, in DAWs if people find the decoded waveform looks like crap, they won't hit the play button, but that's not the case with typical audio players.

https://forum.cockos.com/showthread.php?t=85641
oh my!!!   Most of the files that I have dealt with are +-1.0 floating point, but I do know that it is theoretically possible to sent out files with bigger numbers.  I always believed that +-1.0 floating point is the same as +-32767  for 16 bit signed -- and so be it...  So, you have seen different?   But of course, invalid files with >+-1.0 (assumed that they were invalid) should be clipped and/or flagged.   In between running my Unix pipelines, I do (incorrectly, I believe) SOMETIMES during my experiments,  take advantage of being able to deal with greater than normal floating point values.   For example, I might have an expander whose output might peak at +-2.828 (just as an example) due to exceptional material, but then future normalization stages will fix it.  On the other hand, 'sox' will complain about such files, and probably should process the file if the results are correctly normalized,  or MAYBE SHOULD BASED UPON COMMAND LINE FLAGS simply bounce the file somehow?.

Your point about nonstandard floating point values is definitely a valid issue, however.  (By the way, the term normalized/un-normalized has a specific meaning the the floating point realm -- when the term different 'normalization' was used -- I assumed being able to handle differing/odd ranges of values, right?)
So, to me, it seems like the FP format  should be standardized, or avoided for interchange.  I suspect that for most (all) PRACTICAL purposes, the signed 24 bit should be sufficient, but wonder if it also has such scaling issues?  (I use 24bit only so that I can use flac with reasonably high resolution.)

John Dyson

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #4
Thanks for the warning. And thanks to ReplayGain and fb2k's preamp option that turns down files w/o RG info ...

There could be a lot of different signals in a WAVE file (https://msdn.microsoft.com/en-us/library/windows/hardware/dn653308.aspx - date says 2007, but a version was around before y2k), but I don't think I have seen music published as anything but 16-bits integer, 24-bits integer, whatever-32-bits-that-WavPack-can-contain - and one single IMA ADPCM.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #5
David made a change to WavPack to permit the storage of floating point audio using the same compression method as for integer audio (around the time I was developing TransPCM - which can output Float16 audio from several different inputs).

For Float16, I made use of the full range, i.e. ±65504 at the top end to ±5.96046E-08 as the minimum subnormal, giving a range of 1,098,975,582,421:1 (or approximately 40 bits) with up to 11 significant bits.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #6
A very horrible thing with float 32 file is they can be encoded in different "normalized" fashions. If the decoder misinterpret the format, the result will be horrible full scale square wave-ish like crap which may destroy your equipment. I experienced this nightmare several times, sometimes in foobar2000, sometimes in Audition.

Of course, in DAWs if people find the decoded waveform looks like crap, they won't hit the play button, but that's not the case with typical audio players.

https://forum.cockos.com/showthread.php?t=85641
OK I made a tool to automatically detect these kinds of files, and convert them to the standard formats as shown here. Of course, you need to have these kinds of files to try it.

X

The detection methodology is also somewhat relevant to this thread as well, as my program showed more accurate result than others (SoX, Bitter, Adobe Audition). For example, here is a 8-bit file, with 1dB volume reduction, then dithered to 16-bit.
https://hydrogenaud.io/index.php?topic=120395.0
I search for a tool that can check my 24bit downloads I have purchased over the years. Honestly its just a handful compared to my whole 16bit FLAC collection but I often wondered whether these 24bit files are actually true or just upconverted 16bit.
You could try a bit meter. A few are available as VST plugins, for example Bitter.
X

My result:
X

Download link:
https://1drv.ms/u/s!AvzB71jO7t0-gYwfVGulbuFbVFZcmg?e=uN2p8Z

SHA256:
ee3a919fb267b9afec9494a0ae7f9b3ebd4fad97b8a03cc65245839ec2138d4f

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #7
equipment doesn't get destroyed from its intended use (playing sounds), this is some urban legends.
it can destroy ears if you accidentally play it too loud, though.
some ANC'd headphones + AutoEq-based impulse + Meier Crossfeed (30%)

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #8
Tweeters are almost never capable of outputing maximum power in high frequencies that clipping or integer overflow can create.

A 1 dB volume adjustment with dither can also be detected on a histogram of byte values where some are more likely to occur than others. All bytes of a sample are overlapped on the same chart, but the effect is still clear. This can be run on a fadeout portion to detect smaller discontinuities. But a mild bass boost is enough to make the histogram smooth again.

I was unable to run this program because it requires NET Framework. Would it describe a file with bass boost and phase shift in a way that is detectable? Or in other words, what else besides a volume adjustment can it detect?

Maybe you could output a pseudographic bar chart in the analysis output to get a histogram that is easier to parse by eye (more symbols: greater value).

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #9
I was unable to run this program because it requires NET Framework. Would it describe a file with bass boost and phase shift in a way that is detectable? Or in other words, what else besides a volume adjustment can it detect?
The ReadMe mentioned these things already if you cannot run it. To elaborate, BitSort is used to assist the development of oldsCool so that I don't need to convert everything to .wav files during analysis. It reads the data bytes in files in integer and float with different ranges and guess the format as the header information could be unreliable. So anything resembles a bit meter is just a side effect rather than the main function of the software. For example, BitSort will still detect 16.8 float content if the files are being saved in WavPack as it is lossless and supports both 32-bit integer and float.

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #10
Anyway, here is how the 8-bit RMAA signal looks like with bass EQ, peak normalization (to avoid clipping) and dither. The highlighted parts are still correlated to the original file, not scattered around. Of course, it cannot be compared in this way if there is no reference file, and if you resample, it falls apart anyway.
X

The frequency response of the EQ looks like this:
X

I don't like the presentation of Bitter as the UI is too small and it is pretty hard to count the number of lines, especially when they move quickly.

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #11
OK I made a tool to automatically detect these kinds of files, and convert them to the standard formats as shown here.

Hm. Exposing my ignorance:

* What am I doing wrong when bitsort.exe filename.wav returns "Not enough storage is available to process this command." and I have more than enough space on the drive?

* How do I read the output - in particular, what indicates that I have an Audition-type file where WavPack/OptimFROG should be invoked with the correction option?
I take it that 16/16 for 32-bit floating-point PCM I guess means that someone has taken in a 16-bit file and just exported as float? But what do the two "16" mean, really?

And then this:
"Can't guess the type" and a warning that "output clipped 185095 samples; decrease volume?"
These files - uploader calls them "NON playable 32 bit wav version" :-)
https://soundcloud.com/deceiverdubz/destrudo-free-download
https://soundcloud.com/deceiverdubz/hate-1


And ... maybe a bit off topic, but:
* when a 32-bit file uses only 16 or 24, there should be a way to decimate it "losslessly except for overall volume information" to integer ... but how?

Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #12
* when a 32-bit file uses only 16 or 24, there should be a way to decimate it "losslessly except for overall volume information" to integer ... but how?
If it's normalized to ±1.0, multiply it by 32768 or 8388608 and convert to integer. If it's normalized to some other peak, divide by that number first.

You have to be a little bit more clever to figure out whether you can perform such decimation in the first place and how to do it correctly if so...

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #13
* What am I doing wrong when bitsort.exe filename.wav returns "Not enough storage is available to process this command." and I have more than enough space on the drive?
BitSort does not accept direct file input, it is meant to be used with external decoders.
Quote
* How do I read the output - in particular, what indicates that I have an Audition-type file where WavPack/OptimFROG should be invoked with the correction option?
I am not familiar with WavPack/OptimFROG commands, so can't help.
Quote
I take it that 16/16 for 32-bit floating-point PCM I guess means that someone has taken in a 16-bit file and just exported as float? But what do the two "16" mean, really?
16/16 is SoX's interpretation, not mine. My results are always in the generated txt files. My guess is SoX only sees the used highest bit and lowest bit, without checking anything in-between. My software on the other hand also looks for missing bits in-between. If the input file has a float header, BitSort and oldsCool will always display the results in float format, i.e. not counting bits, just show max/min sample values. For example, 16-bit files disguised as float may often show  1 / 32768 = 3.0517578125e-05 minimum sample value. The same reason that foobar2000's RG scan results often show 0.999969 peak value, which means 32767 / 32768 = 0.999969482421875.
Quote
And then this:
"Can't guess the type" and a warning that "output clipped 185095 samples; decrease volume?"
These files - uploader calls them "NON playable 32 bit wav version" :-)
https://soundcloud.com/deceiverdubz/destrudo-free-download
https://soundcloud.com/deceiverdubz/hate-1
Everything before "BitSort is reading n bytes..." are printed by SoX, not BitSort. As mentioned in the help file in the "Decoder differences" section, for .wav files, oldsCool is the right tool to use, not BitSort. From my observation these files used values up to +1.0, which translates to +32768 or +8388608 or +2147483648 in different integer bit-depth. The last digit is 8 but not 7 and therefore SoX reported clipping when BitSort is used with SoX. With oldsCool, these files should show this:
Code: [Select]
-------------------------------------------------------------------------------
E:\download\DESTRUDO MASTER V2.wav
00:02:20.8000000 = 12418560 samples / 2-ch @ 44100Hz
32-bit floating point
       Ch   Position    Value(+/-)               dBFS
Max    0    28          1                        0
Min    1    4709505     4.085826676991644e-10    -187.77440120645838
-------------------------------------------------------------------------------
E:\download\PAIN.wav
00:02:03.8571428 = 10924200 samples / 2-ch @ 44100Hz
32-bit floating point
       Ch   Position    Value(+/-)               dBFS
Max    1    18914       1                        0
Min    1    5446601     1.0007897621733264e-08   -159.99314292029698
-------------------------------------------------------------------------------
.WAV file(s) processed: 2
foobar2000 uses 32-bit float internal format too, so it can also show the correct values when used with BitSort. foobar2000 however is lossy for 32-bit integer files.

PS: For those who don't know, the updated versions of BitSort and oldsCool are posted here:
https://hydrogenaud.io/index.php?topic=97746.msg1004399#msg1004399

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #14
* How do I read the output - in particular, what indicates that I have an Audition-type file where WavPack/OptimFROG should be invoked with the correction option?
I am not familiar with WavPack/OptimFROG commands, so can't help.

Reason I am asking here is to detect an Audition-type file.

Typing "-a" I can do myself, but fwiw, here is the point:
Quote from: off.exe --help
  --correct_audition  correct Adobe Audition / CoolEdit conversion bug
        when converting from int to float, Audition converts 0 values
        to random noise with maximum relative amplitude < 5*10^-16;
        this option carefully corrects this bug setting them back to 0
        and significantly improves compression ratio in these files
The WAVEFORMATEXTENSIBLE structure is used (if present) to determine the format details. However, there are some programs that use their own non-standard format extensions. The most popular of these is Adobe's Audition (previously Syntrillium's CoolEdit) which created two new 32-bit floating point formats. An option has been added to WavPack (-a) to force the "adobe" interpretation of these floating point formats. If you are compressing integer files do NOT use this option.

Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #15
Here are the 16.8 and 24.0 float files saved with Audition 1.5 so that you can try them with WavPack. Audio data in these two files should be identical to DESTRUDO normal.wav when correctly encoded and decoded.

Bona fide 16.8 and 24.0 float formats were introduced in Syntrillium Cool Edit Pro/2000. 16.8 and 24.0 float files from different versions of Adobe Audition are supposed to show some differences in file headers, so don't take the attached files as the only reference. Installers of old Syntrillium software do not work with modern 64-bit Windows, I used VirtualBox when debugging BitSort and oldsCool but uninstalled now, so can't attach those files at this moment.

Another thing is don't attempt to use oldsCool with CMD, always use drag and drop or SendTo shortcut, otherwise will lose full Unicode path name support. BitSort with foobar2000 is fine with Unicode, as foobar2000 itself can deal with Unicode.

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #16
Wait: so the answer to
"how to check whether a file is of this kind",
is to
"drop it onto oldsCool.exe and see if it converts"
?

Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #17
Look for the 16.8 and 24.0 suffixes when oldsCool is operating in read-only mode (by setting log.txt as read-only). Files with these suffixes will convert when oldsCool is running in normal mode. oldsCool will also change the WAVE_FORMAT_EXTENSIBLE tag (WAVEFORMATEXTENSIBLE is the structure) in non-multichannel (mono/stereo) files so that old Cool Edit Pro can open these files. So if oldsCool displays WAVE_FORMAT_EXTENSIBLE in mono/stereo files, the files will be converted as well, even if they are not 16.8 or 24.0 float files.
Code: [Select]
Press ENTER to operate in read-only mode, other keys to quit.
-------------------------------------------------------------------------------
E:\download\DESTRUDO\DESTRUDO 16-8.wav
00:00:10 = 882000 samples / 2-ch @ 44100Hz
32-bit floating point (16.8)
       Ch   Position    Value(+/-)               dBFS
Max    0    87          1                        0
Min    1    299505      4.085826676991644e-10    -187.77440120645838
-------------------------------------------------------------------------------
E:\download\DESTRUDO\DESTRUDO 24-0.wav
00:00:10 = 882000 samples / 2-ch @ 44100Hz
32-bit floating point (24.0)
       Ch   Position    Value(+/-)               dBFS
Max    0    87          1                        0
Min    1    299505      4.085826676991644e-10    -187.77440120645838
-------------------------------------------------------------------------------
E:\download\DESTRUDO\DESTRUDO normal.wav
00:00:10 = 882000 samples / 2-ch @ 44100Hz
32-bit floating point
       Ch   Position    Value(+/-)               dBFS
Max    0    87          1                        0
Min    1    299505      4.085826676991644e-10    -187.77440120645838
-------------------------------------------------------------------------------
.WAV file(s) processed: 3

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #18
OK. Assume the .wv files are correctly encoded, here is the CMD script to pipe wvunpack to BitSort. Put BitSort.exe, wvunpack.exe and BitSort-wv.cmd in the same location, and send one or more files to BitSort-wv.cmd. Inputs not ended with .wv will be filtered to prevent errors. I did not try OptimFROG but perhaps you can fiddle with the script and try to figure it out.

Make a shortcut of BitSort-wv.cmd in the SendTo folder and send files to the shortcut, otherwise it may not work.
X

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #19
The script above is pretty lame and buggy due to my poor CMD knowledge. After some studies I attached a better one. Files ended with .wv will be decoded by wvunpack, other extensions except .txt will be decoded by SoX, then pipe to BitSort. Piping to FFmpeg could be a good idea as well but the documentation is ridiculously long and hard to follow.

BitSort will still show error if for example, the .wv files are in DSD format. The WavPack manual mentioned -f[n] can be used to check file info without decoding but the results will be sent to stdout. Without modifying the source code of BitSort I don't know how can I use -f[n] in a CMD file so that BitSort can either skip the DSD files or decode to PCM for analysis.

Another thing is while MPC-HC is fine with WavPack DSD, it still produces Merzbow-style sound when playing 16.8 and 24.0 float WavPack files. foobar2000 on the other hand is fine, as well as Adobe Audition, but no surprise because the Audition plugins are downloaded from the official WavPack website.

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #20
MPC-HC, is that forked off to being developed? (Does it use ffmpeg, or ... ? ffmpeg does some strange things every now and then.)
But now I have fought enough codecs for a while, so ... I'm just going to take note that I couldn't understand OptimFrog's handling, and shrug it off.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/


Re: 32-bit floating-point (WAV) files "in the wild"

Reply #22
Despite all the geeky talks on the OptimFROG website, the tools provided are pretty easy to use. Drag integer files (32-bit is supported) to ofr.exe and float files (normal, 16.8 and 24.0 are supported) to off.exe and they will encode, --correct_audition seems unnecessary, at least for the DESTRUDO snippet I posted, but could be useful in case of false positive/negative depending on how OptimFROG deals with these formats.

Decoding on the other hand is not too user friendly as off.exe only works with float and ofr.exe only works with integer, but the encoded files are all ended with .ofr. I have a feeling that the author has a plan to merge these formats and make all of them capable of real time decoding/seeking, but abandoned for years due to some unsolvable difficulties or simply because everyone is using flac now.

The attached script added OptimFROG support. Files ended with .ofr will be decoded by ofr.exe, .off files will be decoded by off.exe.

BTW, Microsoft is officially distributing a stereo 24/96 .wav file in Windows 10 without using WAVE_FORMAT_EXTENSIBLE, so kind of funny. File name is camerashutter.wav but the exact location can be different in different machines. The file has a marker too, and markers can be important for DAW users. flac by default discards these things.
https://www.virustotal.com/gui/file/4ecdcbd3018c03d813978ba453aaecc53ddd178a5b46e82d3c03e11c262403e8

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #23
--correct_audition seems unnecessary, at least for the DESTRUDO snippet I posted
Ah, OK. Explains my headscratching over how to make it matter. Thx.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: 32-bit floating-point (WAV) files "in the wild"

Reply #24
The attached file was generated to trigger the Audition bug, as well as putting --correct_audition into effect.
X
Audacity is showing what the waveform is supposed to be, a distortion free tone, while Audition is showing some funny stuff because it thinks this int32 file is 16.8 float, though oldsCool indicates the file is int32 with 17 empty bits. With --correct_audition, off.exe said it corrected something. However, after correction the file is no longer bit-perfect. Most importantly, it created a lot of distortion in the corrected file.

On the other hand, simply drag the file into ofr.exe, the encoded file will be bit-perfect and much smaller. Because a type 1 32-bit .wav file can also be a Cool Edit/Audition 16.8 float file, both ofr.exe and off.exe can accept these files.

I don't understand what the OptimFROG author wanted to achieve, but given the lossy nature of --correct_audition, this option is of little use. The worst thing is even without using --correct_audition, off.exe still nagged me about this "feature".

A sensible solution is to avoid using a buggy version of Audition to open such files, or use something else like Audacity, or ideally, a 64-bit float DAW (e.g. Reaper) to edit these kinds of int32 files. It sounds strange to me why a software bug should be corrected by changing the audio data of a healthy file, especially for a lossless codec.

Also, I did not know APE also supports int32 now. Apart from insane doesn't work, the Cool Edit/Audition filter provided in the installer also does not support reading and writing int32 APE files. However, since MAC.exe supports int32 and foobar2000 is lossy for int32, the CMD script is updated to support APE. I did not install the whole package and only extracted MAC.exe and the Cool Edit/Audition filter from the installer, so can't comment about other things.