Skip to main content

Topic: R128GAIN: An EBU R128 compliant loudness scanner (Read 244734 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • pbelkner
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #500
I also submitted a bug report to the SoX sourceforge team.  In the meantime you are welcome to use this patch file if you wish.

Many thanks for resolving this and providing the patch.

  • pbelkner
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #501
Since I am using Ubuntu I am unfortuantely stuck with Libav. Trying option number two I get the following error:

This is most likely due to an incompatibility between FFmpeg and Libav (probably a missing method in Libav which is used by r128gain).

When I tried downloading FFMPEG files from here and replacing the files in the r128gain-tools folder I get the following error:

This is completely outdated. "r128gain" uses "libavutil" 52, "libavcodec" 54, and "libavformat" 54.

Via FFmpeg http://ffmpeg.org/download.html I've found the following:
[blockquote]https://launchpad.net/~jon-severinsson/+archive/ffmpeg[/blockquote]

  • pbelkner
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #502
3. The program will not recurse thru directories of 96000 sample rate albums and higher. It crashes immediatley and the window closes in an instant. So if there is a mix of 44100 and 96000 it won't run(on this machine anyway). Or even if only 96000 sample rate albums are in the directory it still crashes.

Unfortunately I'm not able to reproduce this. I did the following:
  • Converted the EBU R128 test vector from WAV to 96 kHz FLAC (using "sox in.wav out.flac rate 96k").
  • Moved the resulting FLACs into two subfolders, "3341" and "3342".
  • Let both, the GTK2 and GTK3 edition run on the root.
  • Everything works as expected:

    Code: [Select]
    SoX sucessfully loaded.
    FFmpeg sucessfully loaded.
    /h/MinGW/msys/1.0/home/home/ebu-loudness-test-setv03-flac-96
    /h/MinGW/msys/1.0/home/home/ebu-loudness-test-setv03-flac-96/3341
      analyzing ...
        [1/12] "1kHz Sine -20 LUFS-16bit.flac": -19.9 LUFS (-3.1 LU)
            peak: -19.9 TPFS, range: 0.0 LU
        [2/12] "1kHz Sine -26 LUFS-16bit.flac": -25.9 LUFS (2.9 LU)
            peak: -25.9 TPFS, range: 0.0 LU
        [3/12] "1kHz Sine -40 LUFS-16bit.flac": -40.0 LUFS (17.0 LU)
            peak: -39.8 TPFS, range: 0.0 LU
        [4/12] "seq-3341-1-16bit.flac": -22.9 LUFS (-0.1 LU)
            peak: -22.9 TPFS, range: 0.0 LU
        [5/12] "seq-3341-2-16bit.flac": -33.0 LUFS (10.0 LU)
            peak: -32.7 TPFS, range: 0.0 LU
        [6/12] "seq-3341-2011-8_seq-3342-6-24bit-v02.flac": -23.0 LUFS (0.0 LU)
            peak: -2.6 TPFS, range: 15.2 LU
        [7/12] "seq-3341-3-16bit-v02.flac": -22.3 LUFS (-0.7 LU)
            peak: -23.0 TPFS, range: 13.0 LU
        [8/12] "seq-3341-4-16bit-v02.flac": -23.0 LUFS (0.0 LU)
            peak: -23.0 TPFS, range: 13.0 LU
        [9/12] "seq-3341-5-16bit-v02.flac": -23.0 LUFS (-0.0 LU)
            peak: -20.0 TPFS, range: 6.0 LU
        [10/12] "seq-3341-6-5channels-16bit.flac": -23.0 LUFS (0.0 LU)
            peak: -24.0 TPFS, range: 0.0 LU
        [11/12] "seq-3341-6-6channels-WAVEEX-16bit.flac": -23.0 LUFS (0.0 LU)
            peak: -24.0 TPFS, range: 0.0 LU
        [12/12] "seq-3341-7_seq-3342-5-24bit.flac": -23.0 LUFS (-0.0 LU)
            peak: -8.9 TPFS, range: 4.8 LU
        [ALBUM]: -23.0 LUFS (-0.0 LU)
            peak: -2.6 TPFS, range: 16.0 LU
      done.
    /h/MinGW/msys/1.0/home/home/ebu-loudness-test-setv03-flac-96/3342
      analyzing ...
        [1/4] "seq-3342-1-16bit.flac": -22.6 LUFS (-0.4 LU)
            peak: -20.0 TPFS, range: 10.0 LU
        [2/4] "seq-3342-2-16bit.flac": -16.8 LUFS (-6.2 LU)
            peak: -15.0 TPFS, range: 5.0 LU
        [3/4] "seq-3342-3-16bit.flac": -20.0 LUFS (-3.0 LU)
            peak: -20.0 TPFS, range: 20.0 LU
        [4/4] "seq-3342-4-16bit.flac": -24.5 LUFS (1.5 LU)
            peak: -20.0 TPFS, range: 15.0 LU
        [ALBUM]: -19.2 LUFS (-3.8 LU)
            peak: -15.0 TPFS, range: 25.0 LU
      done.
    Done.
    Hit enter to continue ...
Code: [Select]
$ uname -a
Linux ubuntu 3.5.0-19-generic #30-Ubuntu SMP Tue Nov 13 17:48:01 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Are you using FLAC as well?

  • Singtoh
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #503
Hello Pbelkner,

Yes I am using flac. Hmm, I will try my little test again and report what happens. I just let it do about 50 normal 44100 flac albums last night and it seems that it worked alright. Is there a way I can output a log, or am I missing where an output log may be being put??. As I said before, I cannot get r128gain to compile and I have no command line capability with r128gain or flac1770, I just run r128gain by clicking the icon in my home directory. It feels like I am missing something here because normally I do a ./configure, make && make install with other programs and everything is usually ok. Whenever I try it with r128gain, it fails immediatly with the same error as I have seen someone post here before.(can't remember the exact error. sorry?). I just compiled Audacity, along with ffmpeg the other day and had no issues?? I'll get back to you after I try this again with a directory full of vinyl rips 24/96 I have.

Cheers,

Singtoh

  • Singtoh
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #504
3. The program will not recurse thru directories of 96000 sample rate albums and higher. It crashes immediatley and the window closes in an instant. So if there is a mix of 44100 and 96000 it won't run(on this machine anyway). Or even if only 96000 sample rate albums are in the directory it still crashes.

Unfortunately I'm not able to reproduce this. I did the following:
  • Converted the EBU R128 test vector from WAV to 96 kHz FLAC (using "sox in.wav out.flac rate 96k").
  • Moved the resulting FLACs into two subfolders, "3341" and "3342".
  • Let both, the GTK2 and GTK3 edition run on the root.
  • Everything works as expected:

    Code: [Select]
    SoX sucessfully loaded.
    FFmpeg sucessfully loaded.
    /h/MinGW/msys/1.0/home/home/ebu-loudness-test-setv03-flac-96
    /h/MinGW/msys/1.0/home/home/ebu-loudness-test-setv03-flac-96/3341
      analyzing ...
        [1/12] "1kHz Sine -20 LUFS-16bit.flac": -19.9 LUFS (-3.1 LU)
            peak: -19.9 TPFS, range: 0.0 LU
        [2/12] "1kHz Sine -26 LUFS-16bit.flac": -25.9 LUFS (2.9 LU)
            peak: -25.9 TPFS, range: 0.0 LU
        [3/12] "1kHz Sine -40 LUFS-16bit.flac": -40.0 LUFS (17.0 LU)
            peak: -39.8 TPFS, range: 0.0 LU
        [4/12] "seq-3341-1-16bit.flac": -22.9 LUFS (-0.1 LU)
            peak: -22.9 TPFS, range: 0.0 LU
        [5/12] "seq-3341-2-16bit.flac": -33.0 LUFS (10.0 LU)
            peak: -32.7 TPFS, range: 0.0 LU
        [6/12] "seq-3341-2011-8_seq-3342-6-24bit-v02.flac": -23.0 LUFS (0.0 LU)
            peak: -2.6 TPFS, range: 15.2 LU
        [7/12] "seq-3341-3-16bit-v02.flac": -22.3 LUFS (-0.7 LU)
            peak: -23.0 TPFS, range: 13.0 LU
        [8/12] "seq-3341-4-16bit-v02.flac": -23.0 LUFS (0.0 LU)
            peak: -23.0 TPFS, range: 13.0 LU
        [9/12] "seq-3341-5-16bit-v02.flac": -23.0 LUFS (-0.0 LU)
            peak: -20.0 TPFS, range: 6.0 LU
        [10/12] "seq-3341-6-5channels-16bit.flac": -23.0 LUFS (0.0 LU)
            peak: -24.0 TPFS, range: 0.0 LU
        [11/12] "seq-3341-6-6channels-WAVEEX-16bit.flac": -23.0 LUFS (0.0 LU)
            peak: -24.0 TPFS, range: 0.0 LU
        [12/12] "seq-3341-7_seq-3342-5-24bit.flac": -23.0 LUFS (-0.0 LU)
            peak: -8.9 TPFS, range: 4.8 LU
        [ALBUM]: -23.0 LUFS (-0.0 LU)
            peak: -2.6 TPFS, range: 16.0 LU
      done.
    /h/MinGW/msys/1.0/home/home/ebu-loudness-test-setv03-flac-96/3342
      analyzing ...
        [1/4] "seq-3342-1-16bit.flac": -22.6 LUFS (-0.4 LU)
            peak: -20.0 TPFS, range: 10.0 LU
        [2/4] "seq-3342-2-16bit.flac": -16.8 LUFS (-6.2 LU)
            peak: -15.0 TPFS, range: 5.0 LU
        [3/4] "seq-3342-3-16bit.flac": -20.0 LUFS (-3.0 LU)
            peak: -20.0 TPFS, range: 20.0 LU
        [4/4] "seq-3342-4-16bit.flac": -24.5 LUFS (1.5 LU)
            peak: -20.0 TPFS, range: 15.0 LU
        [ALBUM]: -19.2 LUFS (-3.8 LU)
            peak: -15.0 TPFS, range: 25.0 LU
      done.
    Done.
    Hit enter to continue ...
Code: [Select]
$ uname -a
Linux ubuntu 3.5.0-19-generic #30-Ubuntu SMP Tue Nov 13 17:48:01 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Are you using FLAC as well?


Hello Pbelken,

Well, I am at a bit of a loss?? The test I ran the other day was run three times each to be sure it failed consistently. Today I have run a mix of a few albums 24bit and 16bit and it ran rite thru. The only thing I did differently was downloaded the FLAC1770 v0.2.0 and extracted it to the r128gain folder in my home folder, weather that has anything to do with it I am not sure?? Here is the output I am getting from terminal now with a mix of albums:

SoX sucessfully loaded.
FFmpeg sucessfully loaded.
/media/Storage/processmusic/12replaygain
/media/Storage/processmusic/12replaygain/2008-07-14 - The Stranger (bonus disc Live at Carnegie Hall 1977) - Sony BMG Music Entertainment  GB
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e28e0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26e29c0] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26bc940] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x26bcb40] max_analyze_duration 5000000 reached at 5015510
[flac @ 0x2748aa0] max_analyze_duration 5000000 reached at 5015510
Consider increasing the value for the 'analyzeduration' and 'probesize' options
  analyzing ...
    [1/12] "08 Band Introductions.flac": [flac @ 0x2748aa0] max_analyze_duration 5000000 reached at 5015510
88.1 dBFS (0.9 dB)
        peak: -1.0 dBFS, range: 12.1 dB
    [2/12] "06 The Entertainer.flac": [flac @ 0x26edc00] max_analyze_duration 5000000 reached at 5015510
94.9 dBFS (-5.9 dB)
        peak: 0.8 dBFS, range: 13.7 dB
    [3/12] "11 Say Goodbye to Hollywood.flac": [flac @ 0x26edc00] max_analyze_duration 5000000 reached at 5015510
97.5 dBFS (-8.5 dB)

a couple of more errors after it ran for about 15 mins.
flac @ 0x2797c60] invalid predictor order: 19 > 0
[flac @ 0x2797c60] decode_frame() failed
[2/9] "06 Rosalinda's Eyes.flac" ... [flac @ 0x2767360] max_analyze_duration 5000000 reached at 5034667
[flac @ 0x2797c60] sample/frame number mismatch in adjacent frames
done.

I don't know of course what the above errors mean?? Anyway, just to report on last nights run. It seems all is ok with those albums but just a little "bug" or glitch happened. Every album was stripped of it's artwork except for the first 2 tracks of each album, and todays runs all the artwork is stripped as is "normal" at this point I guess?? Kinda weird?? Also yesterday, r128gain crashed out of the blue as it was running along nicely, so I restarted it and it ran to the end?? Don't know, please let me know if I can, or how I can get a log other than what is running in the terminal so I can post it for your enjoyment:) I'm not an expert on linux, but I have done a few compilations of kernels and programs and such, so if I can be more helpful, please let me know??

Thanks again,

Cheers,

Singtoh

  • MrMod
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #505
Many thanks for resolving this and providing the patch.


Glad to help the cause (the patch was committed to git on SoX btw).  One thing I noticed in this adventure.  I see that you are upconverting until >= 192k is reached, instead of just doing 4x oversampling.  For 44.1k and 48k material that would be fine (176.4k and 192k respectively).  But what about 24bit/96k material?  I would assume True Peak would require upconverting to 384k to get an accurate reading.  Just curious.
  • Last Edit: 07 December, 2012, 11:11:55 PM by MrMod

  • lvqcl
  • [*][*][*][*][*]
  • Developer
R128GAIN: An EBU R128 compliant loudness scanner
Reply #506
From the pdf file available at http://www.itu.int/rec/R-REC-BS.1770-3-201208-I/en :

Quote
The 4 × over-sampling filter increases the sampling rate of the signal from 48 kHz to 192 kHz. This higher sample rate version of the signal more accurately indicates the actual waveform that is represented by the audio samples. Higher sampling rates and over-sampling ratios are preferred (see Appendix 1 to this Annex). Incoming signals that are at higher sampling rates require proportionately less over-sampling (e.g. for an incoming signal at 96 kHz sample rate a 2 × over-sampling would be sufficient.)

  • pbelkner
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #507
I see that you are upconverting until >= 192k is reached, instead of just doing 4x oversampling.  For 44.1k and 48k material that would be fine (176.4k and 192k respectively).  But what about 24bit/96k material?  I would assume True Peak would require upconverting to 384k to get an accurate reading.

From the pdf file available at http://www.itu.int/rec/R-REC-BS.1770-3-201208-I/en :

Quote
The 4 × over-sampling filter increases the sampling rate of the signal from 48 kHz to 192 kHz. This higher sample rate version of the signal more accurately indicates the actual waveform that is represented by the audio samples. Higher sampling rates and over-sampling ratios are preferred (see Appendix 1 to this Annex). Incoming signals that are at higher sampling rates require proportionately less over-sampling (e.g. for an incoming signal at 96 kHz sample rate a 2 × over-sampling would be sufficient.)

Having based flac1770 on Secret Rabbit Code (SRC) rather then the SoX resampler I have some doubts whether it is possible to compute "true peak" in any reliable way. SRC gives completely different results then SoX even depending on the quality setting.

  • lvqcl
  • [*][*][*][*][*]
  • Developer
R128GAIN: An EBU R128 compliant loudness scanner
Reply #508
AFAIK the passband width of Best Sinc SRC is ~96% which is close to SoX default (95%). So thier peak values should be very close (although not identical).

  • pbelkner
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #509
To fix the issue I created a patch file.

I also submitted a bug report to the SoX sourceforge team.  In the meantime you are welcome to use this patch file if you wish.

I've uploaded a new version using the patch: https://sourceforge.net/projects/r128gain/f...s/r128gain/1.0/.
  • Last Edit: 09 December, 2012, 12:48:26 PM by pbelkner

R128GAIN: An EBU R128 compliant loudness scanner
Reply #510
You might wanna make a separate program for Opus, like you did for FLAC. Although recent FFmpeg understands Opus, your program tags OggOpus files incorrectly. Valid Opus files should be tagged differently using its own recommendation for tagging loudness data. See the spec, Ctrl+F for “R128”.

  • MrMod
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #511
To fix the issue I created a patch file.

I also submitted a bug report to the SoX sourceforge team.  In the meantime you are welcome to use this patch file if you wish.

I've uploaded a new version using the patch: https://sourceforge.net/projects/r128gain/f...s/r128gain/1.0/.

Thank you for the fast turn-around.  r128gain-1.0-alpha-7-5 works great.  Also, thank you for the clarification on 96k material only requiring 2x oversampling.  I sometimes forget that we are dealing with 20-20kHz music, so that makes perfect sense.

  • bandpass
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #512
I have some doubts whether it is possible to compute "true peak" in any reliable way. SRC gives completely different results then SoX even depending on the quality setting.



AFAIK the passband width of Best Sinc SRC is ~96% which is close to SoX default (95%). So thier peak values should be very close (although not identical).


Does the spec say anything about what passband width should be used?

And is there a specific test signal that's proving difficult to determine the true peak for?

  • bandpass
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #513
Okay, from the example coefs in the spec, the transition band is from 5/6 to 7/6 of nyquist, stopband attenuation 40dB, passband ripple 0.1dB.  Here's how to match it with SoX (there's no way to do it with SRC):

sox --plot=octave -n -n upsample 4 sinc -a 40 -t 8k -24k > ebu.m

  • MrMod
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #514
Out of curiosity I tried a long upconvert using:
Code: [Select]
SoX -V -S -r 192000 -b 24 -n -n synth 3:15:00 pinknoise sinc -a 40 -t 8k -24k

and sure enough, the same hang occurs in SoX.  So, I searched through all of their code and found
two more instances in dft_filter.c and tempo.c where SoX will hang like it did for rate.c.  I submitted
a bug report to them and patched my own SoX.  Here is my updated patch file in case you want
to try and use the upsample method bandpass describes:
Code: [Select]
diff -cr sox-14.4.0/src/rate.c sox-14.4.0-update/src/rate.c
*** sox-14.4.0/src/rate.c    Tue Dec 27 22:15:32 2011
--- sox-14.4.0-update/src/rate.c    Tue Dec  4 11:04:47 2012
***************
*** 397,403 ****
    size_t remaining = samples_out - p->samples_out;
    sample_t * buff = calloc(1024, sizeof(*buff));
  
!   if ((int)remaining > 0) {
      while ((size_t)fifo_occupancy(fifo) < remaining) {
        rate_input(p, buff, (size_t) 1024);
        rate_process(p);
--- 397,403 ----
    size_t remaining = samples_out - p->samples_out;
    sample_t * buff = calloc(1024, sizeof(*buff));
  
!   if (samples_out > p->samples_out) {
      while ((size_t)fifo_occupancy(fifo) < remaining) {
        rate_input(p, buff, (size_t) 1024);
        rate_process(p);
diff -cr sox-14.4.0/src/dft_filter.c sox-14.4.0-update/src/dft_filter.c
*** sox-14.4.0/src/dft_filter.c    Wed Mar  2 19:47:52 2011
--- sox-14.4.0-update/src/dft_filter.c    Tue Dec 11 11:25:29 2012
***************
*** 108,114 ****
    size_t remaining = samples_out - p->samples_out;
    double * buff = lsx_calloc(1024, sizeof(*buff));
  
!   if ((int)remaining > 0) {
      while ((size_t)fifo_occupancy(&p->output_fifo) < remaining) {
        fifo_write(&p->input_fifo, 1024, buff);
        p->samples_in += 1024;
--- 108,114 ----
    size_t remaining = samples_out - p->samples_out;
    double * buff = lsx_calloc(1024, sizeof(*buff));
  
!   if (samples_out > p->samples_out) {
      while ((size_t)fifo_occupancy(&p->output_fifo) < remaining) {
        fifo_write(&p->input_fifo, 1024, buff);
        p->samples_in += 1024;
diff -cr sox-14.4.0/src/tempo.c sox-14.4.0-update/src/tempo.c
*** sox-14.4.0/src/tempo.c    Sat Jan 21 16:33:13 2012
--- sox-14.4.0-update/src/tempo.c    Tue Dec 11 11:34:31 2012
***************
*** 151,162 ****
    size_t remaining = samples_out - t->samples_out;
    float * buff = lsx_calloc(128 * t->channels, sizeof(*buff));
  
!   if ((int)remaining > 0) {
!     while (fifo_occupancy(&t->output_fifo) < remaining) {
        tempo_input(t, buff, (size_t) 128);
        tempo_process(t);
      }
!     fifo_trim_to(&t->output_fifo, remaining);
      t->samples_in = 0;
    }
    free(buff);
--- 151,162 ----
    size_t remaining = samples_out - t->samples_out;
    float * buff = lsx_calloc(128 * t->channels, sizeof(*buff));
  
!   if (samples_out > t->samples_out) {
!     while ((size_t)fifo_occupancy(&t->output_fifo) < remaining) {
        tempo_input(t, buff, (size_t) 128);
        tempo_process(t);
      }
!     fifo_trim_to(&t->output_fifo, (int)remaining);
      t->samples_in = 0;
    }
    free(buff);

Unlike the rate effect, beware that sinc automatically adds rate after itself to make the output rate the same as the input rate unless the output rate is specified with a -r switch.  Also, the upsample+sinc is faster than the rate+rate currently being used in the effect chain.
  • Last Edit: 11 December, 2012, 12:22:43 PM by MrMod

  • flloyd
  • [*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #515
Since I am using Ubuntu I am unfortuantely stuck with Libav. Trying option number two I get the following error:

This is most likely due to an incompatibility between FFmpeg and Libav (probably a missing method in Libav which is used by r128gain).

When I tried downloading FFMPEG files from here and replacing the files in the r128gain-tools folder I get the following error:

This is completely outdated. "r128gain" uses "libavutil" 52, "libavcodec" 54, and "libavformat" 54.

Via FFmpeg http://ffmpeg.org/download.html I've found the following:
[blockquote]https://launchpad.net/~jon-severinsson/+archive/ffmpeg[/blockquote]


Thanks for the repsonse, unfortuantely that jon-severinsson link was the same link that I listed and as you stated it is outdated. I tried to find other sources that use "libavutil" 52, "libavcodec" 54, and "libavformat" 54 but unfortunately I couldn't find any linux builds that were all up to date (for example ArchLinux's build is still libavutil 51).

Is there any chance r128gain could be built with MP3 support or are their legal/technical problems preventing that?
Sorry, I have nothing witty to say here.

  • pbelkner
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #516
Is there any chance r128gain could be built with MP3 support or are their legal/technical problems preventing that?

Indeed, I hesitate distributing binary releases including a full FFmpeg build in order to potenially avoid such issues. Anyway, you may consider building "r128gain" yourself.

I've successfully tested the following procedure for building "r128gain" as of today with the latest "r12gain" release (i.e. version=1.0-alpha-7-6) on Ubuntu 12.10 (32-bit) using VMware Player 5.0.1, build-894247, on Windows Vista:
  • Using "Ubuntu Software Center" install the
    • "build-essential",
    • "yasm",
    • "libgtk2.0-dev", and
    • "libgtk-3-dev"
    packages.
  • Download the latest "r128gain" source release:
    • r128gain-<version>-src.tar.gz
    • r128gain-<version>-src-ffmpeg.tar
  • Open a terminal.
  • In the terminal, unpack both archives:
    Code: [Select]
    tar xfvz r128gain-<version>-src.tar.gz
    tar xfv r128gain-<version>-src-ffmpeg.tar
    This will leave you with the directory "r238gain-<version>".
  • Change to the directory "r128gain-<version>":
    Code: [Select]
    cd r128gain-<version>
  • Run "configure" as follows:
    Code: [Select]
    ./configure --prefix=.
  • Run "make":
    Code: [Select]
    make
    Running "make" includes downloading and compiling all needed third party libraries, and it will take a lot of time.
  • If successful, you will be left with the following versions:
    • bin/full/cli/r128gain
    • bin/full/gtk2/r128gain
    • bin/full/gtk3/r128gain
    • bin/tiny/cli/r128gain
    • bin/tiny/gtk2/r128gain
    • bin/tiny/gtk3/r128gain
    The "full" versions possibly are the ones you are looking for. The "tiny" versions are the ones for distribution.

  • nu774
  • [*][*][*][*][*]
  • Developer
R128GAIN: An EBU R128 compliant loudness scanner
Reply #517
I was able to reproduce the hanging issue outside r128gain by running:
Code: [Select]
sox -V -S -r 48000 -b 24 -n -n synth 3:10:00 pinknoise rate 96000 rate 192000

So I dove into the SoX source code and located the problem in the rate effect.  They are improperly casting
size_t to an int which fails at the 2^31 boundary for integers.  To fix the issue I created a patch file.

Although your patch fixes the problem, the actual cause of the problem is not 32bit integer overflow.
Actual problem was quite simple: rate_flush() couldn't be called more than once.
Look at the code:
Code: [Select]
 1  static void rate_flush(rate_t * p)
2  {
3    fifo_t * fifo = &p->stages[p->num_stages].fifo;
4    uint64_t samples_out = p->samples_in / p->factor + .5;
5    size_t remaining = samples_out - p->samples_out;
6    sample_t * buff = calloc(1024, sizeof(*buff));
7
8    if (samples_out > p->samples_out) {
9      while ((size_t)fifo_occupancy(fifo) < remaining) {
10        rate_input(p, buff, (size_t) 1024);
11        rate_process(p);
12      }
13      fifo_trim_to(fifo, (int)remaining);
14      p->samples_in = 0;
15    }
16    free(buff);
17  }

At line 14, p->samples_in is cleared out to zero.
Therefore, next time you enter this function, samples_out (at line 4) will become zero, samples_out - p->samples_out will be negative (at line 5), but it's asigned to size_t (unsigned integer type)....
You see? this was not expected at all.

This re-entrance to draining code could possibly be caused by larger output than the buffer size (so sox cannot drain out whole remaining samples at once).

  • lvqcl
  • [*][*][*][*][*]
  • Developer
R128GAIN: An EBU R128 compliant loudness scanner
Reply #518
Although your patch fixes the problem, the actual cause of the problem is not 32bit integer overflow.
Actual problem was quite simple: rate_flush() couldn't be called more than once.

This re-entrance to draining code could possibly be caused by larger output than the buffer size (so sox cannot drain out whole remaining samples at once).


I tested old version of SoX with the commandline from post #515, and also I changed "synth 3:15:00" to "synth 0:01:00". SoX always tried to drain the rate effect twice, regardless of the number of input/output samples.

The previous code:
Code: [Select]
static void rate_flush(rate_t * p)
{
  fifo_t * fifo = &p->stages[p->output_stage_num].fifo;
  uint64_t samples_out = p->samples_in / p->factor + .5;
  size_t remaining = samples_out - p->samples_out;
  sample_t * buff = calloc(1024, sizeof(*buff));

  if ((int)remaining > 0) {
    while ((size_t)fifo_occupancy(fifo) < remaining) {
      rate_input(p, buff, (size_t) 1024);
      rate_process(p);
    }
    fifo_trim_to(fifo, (int)remaining);
    p->samples_in = 0;
  }
  free(buff);
}


1st call: draining always succeeds, then p->samples_in is set to 0, but p->samples_out isn't.

2nd call: samples_out = 0, so remaining = -p->samples_out. If p->samples_out is less than 2^31 then "(int)remaining" is less than 0 and rate_flush does nothing. But when p->samples_out >= 2^31, the bug occurs.

  • nu774
  • [*][*][*][*][*]
  • Developer
R128GAIN: An EBU R128 compliant loudness scanner
Reply #519
Although your patch fixes the 1st call: draining always succeeds, then p->samples_in is set to 0, but p->samples_out isn't.

2nd call: samples_out = 0, so remaining = -p->samples_out. If p->samples_out is less than 2^31 then "(int)remaining" is less than 0 and rate_flush does nothing. But when p->samples_out >= 2^31, the bug occurs.

Oh, I see. Thanks for the clarification.

  • JJZolx
  • [*][*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #520
What audio codecs does the current 1.0 alpha 7-6 operate on? Is it still only FLAC, WAV and WV, as stated in the first post?

The --help states:
  --overwrite        Overwrite already existing output files.
  --in-place          Overwrite original files (not recommended).

What's the difference? This tool does nothing more than add tagging fields to existing files, correct? That generally doesn't require the file to be overwritten unless a new tag is added.

  • dfavor
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #521
Current r128gain-1.0-beta-src-ffmpeg.tar contains
r128gain-1.0-beta/download/ffmpeg-export-snapshot.tar.lzma which
makes unpacking + building with automatic tools difficult.

Please consider providing a file like
r128gain-1.0-beta-src-ffmpeg.tar.gz or
r128gain-1.0-beta-src-ffmpeg.tar.lzma
to make auto building with normal tools easier.

Thanks.

  • dfavor
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #522
Current r128gain-1.0-beta-src-ffmpeg.tar contains
r128gain-1.0-beta/download/ffmpeg-export-snapshot.tar.lzma which
makes unpacking + building with automatic tools difficult.

Please consider providing a file like
r128gain-1.0-beta-src-ffmpeg.tar.gz or
r128gain-1.0-beta-src-ffmpeg.tar.lzma
to make auto building with normal tools easier.

Thanks.


Ah... I see... the lzma file is drop in the ../download directory and
is unpacked during build.

  • flloyd
  • [*][*][*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #523
pbelkner - Thanks for including the ffmpeg source and instructions. I finally got time to try this again and I was able to build and run R128GAIN succesfully on 64bit Ubuntu 12.10. Only hitch was that it failed to download ImageMagick-6.8.1-3 from SourceForge since that version is now gone. Was able to find it elsewhere though and put it in the proper folder and the build went fine.

Thanks for all the help. It is very appreciated.
Sorry, I have nothing witty to say here.

  • amatala
  • [*]
R128GAIN: An EBU R128 compliant loudness scanner
Reply #524
First I wanted to say thanks for all the hard work which went into developing this great tool - it's very helpful indeed! 

I wanted to use it to set BWF tags to my FLAC files, but as already pointed by other users before, the tool was triggering a full file re-write and messed up some tags like the Vendor string which I did not want to be overwritten.

So instead of using the r128gain tool for writing the tags, I chose to use the "command" option to invoke the metaflac utility and write only the tags I needed without impacting anything else. This even allows to preserver the file time stamp in order to avoid having to back it up again after the change (this info can be easily recalculated in case the file has to be restored).

The command I'm currently using on Windows looks like this:

r128gain "--command=metaflac --no-utf8-convert --preserve-modtime --remove-tag=LoudnessValue --remove-tag=MaxTruePeakLevel --remove-tag=LoudnessRange --set-tag=LoudnessValue=%TLDB:.=% --set-tag=MaxTruePeakLevel=%TPDB:.=% --set-tag=LoudnessRange=%TRDB:.=% \"%TRACK%\"" "c:\temp\test" -o temp

This seems to be working great - I'm currently busy doing some volume testing, but so far - so good!