Skip to main content

Topic: libebur128 - (yet another) EBU R 128 implementation (Read 108040 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • benski
  • [*][*][*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #75
Raiden.

For gated loudness of a segment, the EBU R128 spec requires a 50% window overlap.  Maybe I'm missing something obvious - but I'm not seeing any indication of overlap in the code.  Am I wrong?
  • Last Edit: 06 February, 2011, 02:53:04 PM by benski

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #76
For gated loudness of a segment, the EBU R128 spec requires a 50% window overlap.  Maybe I'm missing something obvious - but I'm not seeing any indication of overlap in the code.  Am I wrong?

The code badly needs some comments, I know.
The lib uses a circular buffer of 400ms (or 3s if mode S is enabled). All 200ms a new gating block is calculated, except at the beginning, where 400ms are needed. This behaviour is controlled by the variable "needed_frames", which is initialized to samplerate*2/5 (line 168) and set to samplerate/5 for all following blocks (line 407).

I did a test with some short samples, logging the number of calculated blocks:
Code: [Select]
$ sox -n sine.wav synth 0.3999 sine
$ ./r128-sndfile sine.wav
global loudness: -inf LUFS

$ sox -n sine.wav synth 0.4 sine
$ ./r128-sndfile sine.wav
1
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.5999 sine
$ ./r128-sndfile sine.wav
1
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.6 sine
$ ./r128-sndfile sine.wav
1
2
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.7999 sine
$ ./r128-sndfile sine.wav
1
2
global loudness: -3.69 LUFS

$ sox -n sine.wav synth 0.8 sine
$ ./r128-sndfile sine.wav
1
2
3
global loudness: -3.69 LUFS
  • Last Edit: 06 February, 2011, 04:18:20 PM by Raiden

  • benski
  • [*][*][*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #77
And in the example of a 600ms input, it should be calculating the loudness for 0-400 ms and then 200-600 ms.  Because in the code it looks like it might be doing 0-400ms and then 400-600ms.  But again I might just be following along wrong - the #define macros made it hard to debug and follow along

  • kode54
  • [*][*][*][*][*]
  • Administrator
libebur128 - (yet another) EBU R 128 implementation
Reply #78
Ah, right, then. Well, then it would be helpful to add modes for retrieving momentary and short-term loudness, gated. Or maybe I'll just stick to using a fork that does what I need. Because using BS.1770 for normalization instead of track or album analysis is probably beyond its design scope anyway.
  • Last Edit: 06 February, 2011, 06:44:54 PM by kode54

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #79
PS An integrated mode live meter basically doesn't do anything different than a track based I-mode calculation. It has start, stop, and reset buttons to delimit the range.

Thanks, I was just curious how that works. It could be interesting to plot short-term and integrated loudness and see how they influence each other.
PPS You don't have to redo the whole calculation each time. The absolute gating threshold does not change and must only be compared once per block. The relative gating threshold must be updated and re-applied after every new block. You may hit a drawback of using plain ANSI C, though. C++ would allow comfortable storing of block energies of virtually infinite number in dynamically growing memory structures like vectors or lists. It's not trivial to match the latter in performance/maintainability by hand coding in plain C. But when you've come so far, I'm sure you are going to find a solution.

I currently do something like this already. All block energies above the absolute threshold are stored in a singly linked list. When ebur128_loudness_global is called, this list is traversed twice, first for the relative threshold and a second time for the gated loudness. And yes, I miss std::list.

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #80
And in the example of a 600ms input, it should be calculating the loudness for 0-400 ms and then 200-600 ms.  Because in the code it looks like it might be doing 0-400ms and then 400-600ms.  But again I might just be following along wrong - the #define macros made it hard to debug and follow along

ebur128_calc_gating_block in line 390 is always called with samplerate*2/5 (400ms) as a parameter. So it will always analyse the last 400ms for each new block. A bit difficult are lines 256-275: I must check if the last 400ms "wrap around" the circular buffer, and handle that case separately.

Another test:
Code: [Select]
$ sox -n sine.wav synth 0.2 sine
$ sox -n noise.wav synth 0.2 noise
$ sox sine.wav noise.wav sine.wav noise.wav sine.wav noise.wav sine.wav noise.wav test.wav
$ ./r128-sndfile test.wav
0.654417
0.654412
0.654412
0.654412
0.654412
0.654412
0.654412
global loudness: -2.53 LUFS

The first block differs from the others. This is probably because the filter states must get filled.

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #81
Ah, right, then. Well, then it would be helpful to add modes for retrieving momentary and short-term loudness, gated. Or maybe I'll just stick to using a fork that does what I need. Because using BS.1770 for normalization instead of track or album analysis is probably beyond its design scope anyway.

How does gated short-term or momentary loudness work? Isn't gating only used for long audio segments where very quiet or silent parts could distort the result?

  • romor
  • [*][*][*][*][*]
libebur128 - (yet another) EBU R 128 implementation
Reply #82
It could be interesting to plot short-term and integrated loudness and see how they influence each other.

It I understand you correctly something like that was already posted in that other R128 thread but without short-term history view: http://www.nugenaudio.com/images/VisLM-H.png
It can also output CSV with all kinds of data, but it's commercial

[edit] actually very commercial, I wasn't aware of it's price until now
  • Last Edit: 06 February, 2011, 09:39:28 PM by romor

  • benski
  • [*][*][*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #83
And in the example of a 600ms input, it should be calculating the loudness for 0-400 ms and then 200-600 ms.  Because in the code it looks like it might be doing 0-400ms and then 400-600ms.  But again I might just be following along wrong - the #define macros made it hard to debug and follow along

ebur128_calc_gating_block in line 390 is always called with samplerate*2/5 (400ms) as a parameter. So it will always analyse the last 400ms for each new block. A bit difficult are lines 256-275: I must check if the last 400ms "wrap around" the circular buffer, and handle that case separately.
The first block differs from the others. This is probably because the filter states must get filled.


I looked at the code again and it's definitely not doing any sort of window overlap.  It's treating each 200ms block of audio separately, except for the first block which is done as 400ms.  We should get this fixed.  Unfortunately, the IIR filter code is writing the filtered data back into the audio_data buffer.  In order to do overlapped windows, you'd need to keep the audio intact as each sample will be used twice.  One simple solution would be to do the sum-of-squares inside the filter function, so that you don't need to store the filtered audio.

Just to be clear, this is essentially what needs to be done for overlap.  For the sake of clarity, let's use a 1000Hz sampling rate.
Code: [Select]
ebur128_filter_double(st, &audio_data[0], 400);
ebur128_filter_double(st, &audio_data[200], 400);
ebur128_filter_double(st, &audio_data[400], 400);
ebur128_filter_double(st, &audio_data[600], 400);
/* etc. */


  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #84
It is doing window overlap. Otherwise, the block energies from the test in post 81 would be different, and not all 0.654412.

  • kode54
  • [*][*][*][*][*]
  • Administrator
libebur128 - (yet another) EBU R 128 implementation
Reply #85
The library doesn't like it very much if the sample rate isn't evenly divisible by 2.5 or 5. Everything works fine if I change every *2/5 to a /5*2.
  • Last Edit: 07 February, 2011, 09:11:29 AM by kode54

  • pbelkner
  • [*][*][*][*]
libebur128 - (yet another) EBU R 128 implementation
Reply #86
/5*2.

It's the way lib1770 is doing it as well, cf. "bs1770_stats.c", lines 66, 67.

  • benski
  • [*][*][*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #87
It is doing window overlap. Otherwise, the block energies from the test in post 81 would be different, and not all 0.654412.

Oh ok I see what it's doing now.  It's doing the windowing in ebur128_calc_gating_block instead of ebur128_add_frames_/ebur128_filter_.  Sorry for making such a stink, just wanted to be sure that the library was working correctly.  I had gotten concerned when it was running the filter on 200ms blocks but now I see that it's doing the sum-of-squares on 400ms segments.  Sorry for the alarm

  • nucelar
  • [*]
libebur128 - (yet another) EBU R 128 implementation
Reply #88
Hi raiden, you're last improvements/ additions are awesome!
Good work!

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #89
Hi again,
I've uploaded 0.2.0:

Library:
- Removed the function ebur128_set_channel_map. Use ebur128_set_channel instead.
- Fixed the samplerate bug (the library would not work correctly with a samplerate that was not divisible by 5).
- ebur128_*_multiple functions skip NULL pointers.

Scanners:
- Removed the Python tagging script. Tagging is now directly integrated into the scanners. I'm using TagLib for that. Usage example:
Code: [Select]
r128-sndfile -t album foo/ (tags files in the folder "foo" as an album)
r128-sndfile -t album -r bar/ (recursively tag sub-folders of "bar", each sub-folder as an album)
r128-sndfile -t album foo.ogg bar.ogg baz.ogg (tag files as an album)
r128-sndfile -t track foo.ogg bar.ogg baz.ogg

This should now provide much of the functionality of similar commandline tools like vorbisgain or mp3gain.

Download

  • habasud
  • [*]
libebur128 - (yet another) EBU R 128 implementation
Reply #90
Hi again,
I've uploaded 0.2.0:
[...]


Hi Raiden,

first of all thanks for making and publishing such a useful peace of software! What a lucky day it was when I saw it compiling on FreeBSD without errors...   

With the latest version I noticed that a script I wrote for normalizing didn't work any more. It turned out that the text output of r128-ffmpeg changed somewhat. You can see it when you pipe the output to e.g. hexdump. So in my case piping the output to bc for calculation causes a syntax error.

And a suggestion: As I said I use your software for calculation of normalization. When it turns out that an audiofile has to be made "louder" I measure the given headroom with sox - actualy another time costing scan. Maybe it would be useful if you add a switch that outputs the highest true-peak within a given file.

This is just meant as a feedback - thank you!

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #91
It turned out that the text output of r128-ffmpeg changed somewhat. You can see it when you pipe the output to e.g. hexdump. So in my case piping the output to bc for calculation causes a syntax error.

Yes, the output changed in the last version a bit... Currently it is not really meant for scripts as it writes everything to stderr. I will try to write something useful to stdout.
And a suggestion: As I said I use your software for calculation of normalization. When it turns out that an audiofile has to be made "louder" I measure the given headroom with sox - actualy another time costing scan. Maybe it would be useful if you add a switch that outputs the highest true-peak within a given file.

I will try to get true peak measurement done for the next version, thanks for the feedback.

  • C.R.Helmrich
  • [*][*][*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #92
If I don't reply to your reply, it means I agree with you.

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #93
I've uploaded 0.2.1.

Library:
- removed the ebur128_start_new_segment and ebur128_loudness_segment functions. To calculate track/album values use one library state for each track, then calculate the album gain with ebur128_loudness_global_multiple.
- documented ebur128.h with Doxygen.

Scanner:
- fixed a crash with the FFmpeg scanner.
- fixed recursive folder scanning (didn't work in 0.2.0).
- implemented true peak scanning! Those values will NOT be used for the ReplayGain tags.
- improved scanner output a lot. Hopefully it is now easier to parse by scripts.

The scanner can now calculate all three values recommended by the EBU: "Programme Loudness", "Loudness Range" and "True Peak Level".
Quote
$ ./r128-sndfile -l -p true ~/music/bad\ loop\ -\ Luo/*.flac

-12.81 LUFS, LRA: 14.16 LU, true peak: 0.99826229, /home/jan/music/bad loop - Luo/bad loop - Luo - 01 Nio.flac
-11.15 LUFS, LRA: 8.26 LU, true peak: 0.99095666, /home/jan/music/bad loop - Luo/bad loop - Luo - 02 Eri Valeire.flac
-10.14 LUFS, LRA: 11.79 LU, true peak: 0.99171823, /home/jan/music/bad loop - Luo/bad loop - Luo - 03 Kauniit Ihmiset.flac
-11.31 LUFS, LRA: 11.75 LU, true peak: 0.92898595, /home/jan/music/bad loop - Luo/bad loop - Luo - 04 Mmin.flac
-26.13 LUFS, LRA: 14.87 LU, true peak: 0.25203928, /home/jan/music/bad loop - Luo/bad loop - Luo - 05 3b Or T.flac
-14.10 LUFS, LRA: 11.40 LU, true peak: 1.02603507, /home/jan/music/bad loop - Luo/bad loop - Luo - 06 Kannas Nsp.flac
--------------------------------------------------------------------------------
-11.75 LUFS, LRA: 13.34 LU, true peak: 1.02603507


A note regarding true peak scanning: Even with 4x oversampling it is still possible to underread the "true" peak value by ~0.6 dB. The EBU therefore recommends a "Maximum Permitted True Peak Level" of -1dBTP (1 dB below 0dBFS).

Download

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #94
And here is 0.2.2, with a single change:
- added "--gate" option to specify the relative gate. The standard uses a gate of -8 dB, but will switch to -10 dB in the next revision. This option is for testing only and will be removed when the standard is updated!

I've scanned my FLAC library again with the new gate:
Code: [Select]
r128-sndfile --gate -8.0 *.flac: -10.52 LUFS
r128-sndfile --gate -10.0 *.flac: -10.77 LUFS

-> a difference of 0.25 dB.

  • nucelar
  • [*]
libebur128 - (yet another) EBU R 128 implementation
Reply #95
And here is 0.2.2, with a single change:
- added "--gate" option to specify the relative gate. The standard uses a gate of -8 dB, but will switch to -10 dB in the next revision. This option is for testing only and will be removed when the standard is updated!


Hey Raiden you are great!!! Not only are you on the cutting edge of the spec but also do it all well. FYI I've tested libebur128 with the test set provided by Qualis Audio
http://www.qualisaudio.com/downloads.htm
and ALL test have passed perfectly!!! (except test #2 which is ac-3 encoded 5.1 and I'm not sure if libebur128 can handle this)
Just a couple of remarks:
-It should be sufficient to indicate 1 decimal point in dB figures. For example, some test that should measure -23.0 LUFS according to Qualis, measure -22,97 LUFS in libebur. Of course it is the same but it just doesn't look so good  And I think EBU Tech docs use 1 decimal point always.
- Please make the True Peak measurement in dBTP!!! At first I thought the measurement was wrong (0,793..) but of course this is linear for -2.0 dBTP !! Good work!!!

On the other hand:
Could someone guide me through the steps of how to build an executable for OSX from source code?  is it complicated? Does it imply modifying the code? I have XCode (mainly because of some bundled apps) but I have no idea where to start 
Best regards!
Carles

  • Raiden
  • [*][*][*]
  • Developer
libebur128 - (yet another) EBU R 128 implementation
Reply #96
Hey Raiden you are great!!! Not only are you on the cutting edge of the spec but also do it all well. FYI I've tested libebur128 with the test set provided by Qualis Audio
http://www.qualisaudio.com/downloads.htm
and ALL test have passed perfectly!!! (except test #2 which is ac-3 encoded 5.1 and I'm not sure if libebur128 can handle this)
I'm glad that it works for you!
I've had a quick look at LM2.wav, and it seems to be just a normal stereo 24-bit PCM wave file at 48000 Hz. They say in TechNote #2:
Quote
This signal is Dolby Digital encoded and
stimulates all channels simultaneously, including
the LFE.
I don't have any idea what they mean.
Also, does anyone know why they use a 75% overlap instead of a 50% overlap?

-It should be sufficient to indicate 1 decimal point in dB figures. For example, some test that should measure -23.0 LUFS according to Qualis, measure -22,97 LUFS in libebur. Of course it is the same but it just doesn't look so good  And I think EBU Tech docs use 1 decimal point always.
You're right. I've been using two digit precision because it's a bit more helpful in debugging.
- Please make the True Peak measurement in dBTP!!! At first I thought the measurement was wrong (0,793..) but of course this is linear for -2.0 dBTP !! Good work!!!
I will add an option for displaying the peak in dBTP.

Could someone guide me through the steps of how to build an executable for OSX from source code?  is it complicated? Does it imply modifying the code? I have XCode (mainly because of some bundled apps) but I have no idea where to start 
I don't know much about OSX, but you'll probably need to use some kind of ports system like MacPorts.
A command like "sudo port install libsndfile taglib cmake glib2" should install all needed dependencies to build the r128-sndfile scanner with the instructions in the README.

  • nucelar
  • [*]
libebur128 - (yet another) EBU R 128 implementation
Reply #97
Hi!
Thanks for the Macports suggestion, i will try it.
File #2 is most probably a Dolby-E encoded 5.1 signal. Dolby-E is a broadcast codec that compresses and encodes 5.1 to a stereo bitstream. Decoding needs expensive hardware or software. I wouldn't care about this.
Cheers

  • habasud
  • [*]
libebur128 - (yet another) EBU R 128 implementation
Reply #98
...maybe someone finds it useful to visualize the output of r128-ffmpeg (-m/-s/-i). I wrote a simple shellscript that passes the data to gnuplot:
http://home.arcor.de/claus.misfeldt/shared/r128_plot.tar.gz

Thank you Raiden for great work!

  • Surfi
  • [*][*][*]
libebur128 - (yet another) EBU R 128 implementation
Reply #99
::

SSE2 Win32 compile doesn't work with command
"r128-sndfile.exe" -p true *.flac

OS is Win XP SP3 with all updates. Non-SSE build has no problems.


Greetings ...

::