Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Large output file size after re-encoding (Read 4645 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Large output file size after re-encoding

Hi!

A have a flac file that is encoded by an unknown flac encoder. It verifies without integrity errors in foobar2000. When it gets re-encoded with the official x32 1.3.2 build, about 20% larger file size is produced. Similar results are produced by the 2018/11 x64 1.3.2 build from rarewares.org as well. The difference in file size implies that a bug exists in the reference encoder that prevents efficient encoding in cases like this one. It could also be that a trait of the lossless file is exploited that leads to the efficient encoding. It would be for the benefit of the official implementation to exploit this trait as well, if this is the case.

I have also reported this to the sourceforge flac project, but I don’t see much activity going on, there. I am not aware of the original encoder to use for creating a short sample.  So, the whole file will be here for a limited period [link removed] If this is not allowed here, I can remove the link and provide it to you with PM.

Re: Large output file size after re-encoding

Reply #1
Different FLAC encoders are tuned slightly differently. Also, efficiency of compression depends of source material. On this material used encoder is better, but it can be worse on other material. For example ffmpeg's FLAC  encoder with -compression_level 8 compresses your file even more effectively. but on common 16/44 material it is usually slightly worse than reference 1.3.2 encoder.

Re: Large output file size after re-encoding

Reply #2
I can understand that a difference between 1% and 3% is justified by each encoder’s tuning or implementation logic. It is something quite common. The problem with this case is the huge difference of 20% size increase. This is not common, and I think is a huge opportunity for investigation that can lead to performance increase. 96/24 files are becoming quite common in digital sales and reference FLAC’s performance is poor.

 

Re: Large output file size after re-encoding

Reply #3
I downloaded this file from a link in SF bug tracker. The spectrogram shows that it has nothing above 24 kHz (apparently the audio was upsampled from 48 to 96 kHz). And yes, the reference FLAC encoder doesn't work well with such files.

Re: Large output file size after re-encoding

Reply #4
If that is the case, it could be easy to fix. Some official hi-res releases are actually the outcome of upsampling, and the reference FLAC should perform in such cases. I guess you can upsample from 48k to 96k, and then go back to 48k, losslessly. So, that is an easy shortcut to improve the encoder’s performance. You just compress the 48k version or something similar to it.

Re: Large output file size after re-encoding

Reply #5
If that is the case, it could be easy to fix. Some official hi-res releases are actually the outcome of upsampling, and the reference FLAC should perform in such cases. I guess you can upsample from 48k to 96k, and then go back to 48k, losslessly. So, that is an easy shortcut to improve the encoder’s performance. You just compress the 48k version or something similar to it.

You can't go from 48 KHz to 96 KHz and that back to 48 KHz losslessly unless you used something like zero order hold during those processes (which will produce a ton of aliasing in the 96 KHz file).  At least from the testing I did on various resampling techniques.  You're not actually doubling every sample and than taking away every other sample with all the other types of resampling but rather interpolating totally new samples in between points or using a single new sample to represent multiple points.  That aside any good resampling will always be transparent at anything around 44.1 KHz or higher typically (nothing that should be audible or noticeable) unless your equipment is very poor at handling ultrasonic frequencies from the stuff in some of the more common sample rates above 48 KHz such as 96 KHz for example.

Re: Large output file size after re-encoding

Reply #6
I guess you can upsample from 48k to 96k, and then go back to 48k, losslessly. So, that is an easy shortcut to improve the encoder’s performance.
This would then amount to a new incompatible format that would no longer be FLAC. With the resampling, it would be slower to decode, and the resampler would have to be precisely defined to produce same output on all systems.

I tried to downsample this track with SoX, and encode the result and the delta with TAK and the results were slightly worse than direct encoding. TAK already handles upsampled content very well and is fast.

TAK .. 72,362,475 bytes
TAK48 58,900,454 bytes + takDelta 17,828,895 bytes = 76,729,349 bytes
TAK48_22bit  50,468,215 bytes + takDelta_22bit 23,957,422 bytes = 74,425,637 bytes

Re: Large output file size after re-encoding

Reply #7
Thank you for the responses. Let’s hope that the project is more active than it seems, since there is a bit more space for improvement.

Re: Large output file size after re-encoding

Reply #8
Shoot, that was a big difference.  Motörhead Deaf Forever DVD-Audio edition, made a 6ch .wav out the first 1GB .AOB file and tested:

5876 kbps with TAK -p4m
6642 kbps with WavPack High -x6 (I didn't bother Extra High)
6785 kbps with ffmpeg-created FLAC using -compression_level 8
7791 kbps with reference FLAC (1.3.1) -8.  Using -8 -p -e improves only to 7781.
(13824 uncompressed)

And yes, the reference FLAC encoder doesn't work well with such files.
Are there better settings for these particular files?

Re: Large output file size after re-encoding

Reply #9
so far the observation is that ffmpeg works better for high sample rate content and reference flac works better for CDDA resolution content?
a fan of AutoEq + Meier Crossfeed