I'm performing some topic necromancy to let this thread come to a conclusion.
I recently re-ran the test here, I mostly did this out of my own curiosity.
But I also felt that John Siau, 2bdecided, bandpass, and others that contributed to this thread deserved some more answers.
The plot shown by bandpass showed several tracks with very high Intersample Peaks aka True Peaks.
Apologies that I do not have a tool others can easily use, but I'll describe what I did instead.
I created a tiny program that acted as a custom CLI encoder for Foobar2000, the only thing it did was pipe the audio from foobar2000 to sox and parse and log the output.
I also configured foobar2000 to put the artist/album/track/trackno/replaygain gain/peak info as part of the file name, so that my tiny tool could parse/add that to the log as well.
The latest foobar2000 version was used (v1.3.9), and all tracks was rescanned with the replaygain scanner in v1.3.9 with the scanner set to 2x oversampling for peaks (anything higher seemed to report the same peaks so 4x oversampling as suggested in EBU R128 specs is not possible). The peaks reported by SOX is thus used instead, I also confirmed visually using Adobe Audition 3 is the wave peaks do reach the oversample peak values reported by SOX.
The true magic in the test is due to SOX (big thanks to bandpass for giving me clues to how to use the the command line properly to do this).
The SOX command line used is the following: sox.exe - -n stats upsample 4 sinc -a 40 -t 8k -24k stats
The audio piped from foobar is piped as 32bit (with wav header).
I had to run 4 instances dividing the workload of 6528 tracks, otherwise it would have taken over 16 hours to run this scan; instead it only took around 6 hours (causing 80% load on a 6 core CPU).
Here are some stats from a little helper program I created to crunch some numbers from the CSV log:
6528 tracks.
Avg. Peak -1.24 dBFS (Min -24.55, Max 0.00)
Avg. ISP -0.56 dBFS (Min -25.71, Max 9.54)
ISP/Peak Delta -1.79 dB (Min -49.06, Max 9.54)
RMS -16.79 dBFS (Min -42.05, Max 0.00)
24.08% (1572) ISP <-1 dBFS
75.92% (4956) ISP >-1 dBFS
18.57% (1212) ISP -1 to 0 dBFS
43.49% (2839) ISP 0 to 1 dBFS
8.70% (568) ISP 1 to 2 dBFS
1.53% (100) ISP 2 to 3 dBFS
0.57% (37) ISP 3 to 4 dBFS
0.17% (11) ISP 4 to 5 dBFS
0.23% (15) ISP 5 to 6 dBFS
0.46% (30) ISP 6 to 7 dBFS
1.44% (94) ISP 7 to 8 dBFS
0.75% (49) ISP 8 to 9 dBFS
0.02% (1) ISP 9 to 10 dBFS
0.00% (0) ISP 10> dBFS
Immediate conclusion is that 10 dB headroom is needed to avoid any Intersample Peaks or True Peaks from causing distortion (if the distortion is audible or not is a different discussion).
A few notes on the tested tracks. They are my personal collection, collected over many years. They are MP3, Ogg, AAC, FLAC formats/encodings. Some of it is mainstream. Some of it Iv'e composed myself (3 published albums plus some extra released and non-released stuff).
Some of the tracks are not main stream, rare or not purchaseable. Some have been ripped by me from games (as no official soundtrack existed).
In other words it's a weird even eclectic mix of tracks and sourced stuff. Thus it may or may not be representative of the common listener. Then again people or audiophile or engineers/techs that know or worry about intersample peaks are that common either, certainly not mainstream.
I'll list specific problem tracks if they can be found on the net (legally) somewhere or instructions on how/where they can be found (you may need to rip it from a game/source yourself for example). If a series of related tracks have high True Peaks then I'll list the artist/source/album so you can search for it yourself. Also note hat due to how releases are there may be differences on which year/region the release was made in/for.
A few years ago I copied/encoded all my CDs and recycled all my CD covers and CD inlays and threw the discs in the trash (can't be recycled, at least not at the time) so I no longer have proof of purchase for them (in retrospect kinda stupid, I could have put the CD inlays in a box somewhere) so I'm not giving a full list of the tested collection because of that (sorry). Also not all of them are FLAC (many years ago I ripped most of my music, I never got around to re-ripping it all as FLAC, drive space was not that cheap back then) so the encoding may be the source of the intersample peaks rather than the mastering of the track, I'll mention if this is the case.
The only track that had a True Peak above +9 dBFS:
Jayce and the Wheeled Warriors, opening theme (mp3) +9.54 dBFS, -11.88 RMS
I can't recall where it's from, I think I ripped this from from a Youtube video. I watched this show when I was young, so it's in my collection for nostalgia reasons. Can't share the track for legal reasons, sorry. But it should be searchable on Youtube so try there first (you'll also be treated to a cheesy 80s animated intro as well).
Quite a few tracks have surprisingly high intersample peaks that are above +8 dBFS.
Legendary standup comedian George Carlin's performances/recordings/albums "Back in Town", "Complaints And Grievances", "Playin With Your Head" all have true peaks at/above +8.0 dBFS, the RMS varies from -17 dBFS to -21 dBFS, so even if EBU R128 or ReplayGain was used the true peaks would still be above 0 dBFS afterwards.
Michael Land's Monkey Island III OST and RockStar Games Grand Theft Auto Liberty City Stories OST also have very high true peaks, their RMS is higher than George Carlin's stuff for various reasons (talking vs music being one, but also the years they where mastered, and the way they where mastered).
Liberty City Stories OST should be somewhat available (check WIMP, Spotify, iTunes, Google Play, Amazon etc) but I'm unsure if they match the game audio rip or not. (the in-game radio channels are sometimes mastered differently from the individual tracks).
Frank Klepacki's Blade Runner The Game soundtrack also show similar high true peaks.
His website is at http://www.frankklepacki.com/ you can find some tracks in the flash player on the page http://www.frankklepacki.com/portfolio/game-BR.html
You'll have to rip the tracks from there (or use a live True Peak meter), the rest of the tracks you'll have to rip/convert from Blade Runner The Game itself.
The majority of the other tracks with true peaks above +3 dBFS was also the from the same collections as those above.
(This explains the high anomaly on bandpass's plot/chart that 2bdecided was curious about.)
A few other collections are above +3 dBFS though.
A few tracks from the "Trilogy" album by "Carpenter Brut" (mp3) for example (check the various outlets or bandcamp or Youtube https://carpenterbrut.bandcamp.com/album/trilogy )
A few tracks from Yuki Kajiura's "Noir" soundtrack OST2 disc (mp3). Eminem's album "Relapse" (mp3). Savant's album "ISM" (FLAC)
Tracks with True Peaks in the +2 dBFS to +3 dBFS range.
Eminem's album "Relapse", Type O Negative's "World Coming Down" album, Pendulum's album "Immersion", Savant's "ISM" album, Vader's "XXV" album, Ramin Djawadi 's season 1 "Game of Thrones" soundtrack). Some of this stuff is more mainstream so tests should hopefully be more easily reproducible by others.
A note to John Siau:
Your DAC headroom of +3.5 dBFS seems to be a pretty good choice. Although in some few cases like Eminem's album "Relapse" it's just barely enough. And with artists that like Savant (ISM album) or Carpenter Brut (Trilogy album) they actually break that +3.5 dBFS headroom. If that is audible or not is another matter. Savant's ISM album actually has true peaks above 0dBFS as flac. The title track "Ism" from the album has a true peak of +4.20 dBFS, though the RMS is at +8 dBFS so EBU R128 or Replaygain would bring the true peak to below 0 dBFS in this case luckily, the track Mystery has a true peak of +3.59 dBFS.
Closing notes:
For the most part it seems that MP3 or AAC lossy encoding is a common trend among these.
The only exception is Savant's album ISM which has very high true peaks even as lossless FLAC, the current official place to get the album seems to be at http://savantofficial.bandcamp.com/album/ism the official website (and the shop there) seems to have issues currently, the preview there is lossless though no idea if the true peak is the same or worse with that though.
For those curious the particular track in question clips a lot and has lots of 8bit type of sounds in it. The rest of the album is similar.
The track Ism (from thew album of same name) if upsampled from 44.1kHz to 192kHz ends up with a max peak at +5.41 dBFS and over a million possibly clipped samples (or so states Adobe Audition at least.) That track be be worth using as a test for DACs or upsampling or True Peak testers as it satisfies the criteria of being in the wild and a real world example (rather than the artificial one as linked to in the first post of this thread).
I hope this rather long post is of some use to people out there.
It would be nice if someone could make a True Peak scanner tool (maybe base it around EBU R120 and libsoxr?) either as a foobar2000 plugin or a standalone tool (that you can easily pipe audio to) and log True Peak and RMS and other stats in a convenient file that people could upload/put online.
As I said earlier, if one can hear True Peaks above 0 dBFS or not is another discussion, but I can confidently state that True Peaks above 0 dBFS certainly do not improve the sound quality. And when music do exist in the wild that some of the current very high headroom DACs do not handle then that is at the very least of interest for further study. Should DACs have +6 dBFS headroom, or maybe +10 dBFS headroom? A +10 dB headroom would handle my entire collection without any clipping, then again I use Foobar and Replaygain so peaks never go that high anyway.
Except for those George Carlin recordings, if I apply replaygain to those then for one of them the gain is +8.19 pushing the true peak from +5.19 to +13.6 dBFS, now the digital peak is at -2.94 dBFS so at most the applied gain would be +2.94 which would make the true peak +8.13 which is still a lot. Would +10 dB headroom be enough for "all cases" then?
Note! As I've changed my digital "life" some time ago (new email etc. yay almost 0 spam now) I'm not using this email/account anymore. Hence the reason I wanted to follow up on this thread. If may be posting under a new nick in the future (no idea when, I've hardly been active at all on HA for a long time now, we'll see I guess).
So this will probably be the last post by this user/nick.