Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Can lossy encoding create audible clipping not present in the source? (Read 9920 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Can lossy encoding create audible clipping not present in the source?

Occasionally I find I have a CD with audio which clips or is on the edge of clipping, from which lossy encoding does cause or aggravate clipping.


And can you ABX those differences?
I think this aggravation is not listenable at all...

Can lossy encoding create audible clipping not present in the source?

Reply #1
Occasionally I find I have a CD with audio which clips or is on the edge of clipping, from which lossy encoding does cause or aggravate clipping.


And can you ABX those differences?
I think this aggravation is not listenable at all...


I'm not talking about some subtle artefact such as pre-echo or ringing or some other difference that an untrained ear might not notice, I'm talking about audible clipping i.e. gross distortion that sounds terrible and makes my headphone housing physically vibrate while the speaker distorts.  If you'd like to set the religious police on me I suppose there is nothing I can do, but clipping is a well known artefact which can arise from lossy encoding http://wiki.hydrogenaudio.org/index.php?title=Clipping when the source is already at or approaching the threshold of distortion.

Quote
Lossy audio encoding and decoding can cause the highest/lowest sample values to go over the allowed limit (in practice having the sequential max/min values), which may lead to clipping seen by analysis tools, or even audible clipping. But whether the clipping is truly audible or not is a totally different thing. There are different methods to avoid clipping in lossy audio. Look at the specific audio format answers how to best avoid clipping in each case.


It's not that common that the distortion is audible even with brickwall mixes but it isn't exactly unheard of either.  The last time I encountered it was not on a typical modern brickwall mix, but on a recently acquired CD "Frühe Englische Orgel Musik", part of a well known box set from EMI from 2001 (http://www.medieval.org/emfaq/cds/emi26522.htm).  The engineers used the full dynamic range of the medium to within a whisker of the limit, and lossy compression takes it very audibly over the edge and into the area where you immediately shutdown the playback or unplug the speakers to prevent damage.

I've learned to appreciate that when "conversing" with people whose mind is already fixed that there is a menu, and the next item on the menu is often an accusation that the "heretic" is referencing a pirated lossy, transcoded source and so is not in fact a human being but simply an aggregation of ghastly moral defects....but anyway here is a picture of my CD that I just took a few minutes ago 



I'm sorry but am unwilling to perform an abx test on a sample that is unamibiguously audibly clipping and can damage my headphones or speakers.  I will happily abx this on my Sony MDR ZX700 headphones or Sennheiser CX95 earphones on receipt of £65 or equivalent, or on any headphone that anyone cares to provide at no charge.  What I can offer without again risking damage to my headphones is an image of the waveform of the offending part of the track (03 - William Blitheman - Gloria tibi Trinitatis 6) in audacity uncompressed, and in Ogg Vorbis -q 7 and from wavpack:

uncompressed wav:



ogg -q 7:



from wavpack -chb2.25 lossy (.wvc ignored)



I do know the rules about TOS#8 and still there is no way I am putting this through my speakers or phones again.  If you still have some religious belief that audible clipping isn't a possible outcome of high quality lossy compression then you are welcome to a sample of the relevant portion of the wav, which is a 4 minute section of about 40MB, by offering me somewhere to upload it at your convenience, and you can transcode it and play back the results through your own speakers at your own risk and do your own abx.


Can lossy encoding create audible clipping not present in the source?

Reply #3
I'm sorry but am unwilling to perform an abx test on a sample that is unamibiguously audibly clipping and can damage my headphones or speakers.  I will happily abx this on my Sony MDR ZX700 headphones or Sennheiser CX95 earphones on receipt of £65 or equivalent, or on any headphone that anyone cares to provide at no charge.


For a modern DAC, clipping shouldn't damage your equipment due to the reconstruction filter.  At most it'll boost the volume a dB or two, probably not even for such mild clipping as you have in those images. 

Can lossy encoding create audible clipping not present in the source?

Reply #4
What I can offer without again risking damage to my headphones is an image of the waveform of the offending part of the track (03 - William Blitheman - Gloria tibi Trinitatis 6) in audacity uncompressed, and in Ogg Vorbis -q 7 and from wavpack:

When a lossy encoding is played, the decoded audio samples must be clamped to the range of the target bitdepth (e.g. 16 or 24 bits, generally) because the error introduced by the lossy encoding can cause samples that almost clipped previously to move outside target value range. If this is done correctly, the result should be inaudible (or at least very subtle) in most cases.

However, if there is a bug in this process, or it is skipped altogether, you might get samples that wrap instead of clip. This is extremely audible (can be a loud “blatt”) and is probably what you are hearing.

The funny thing is that there is a bug in the WavPack FFmpeg decoder that causes this on WavPack lossy files, and when I first read your post I thought you were talking about that, but I see now that you saying that WavPack is the codec that is not doing this (obviously you are not using FFmpeg, or someone fixed that). However, it may that the decoder you are using for Ogg has a similar bug.

Can lossy encoding create audible clipping not present in the source?

Reply #5
For a modern DAC, clipping shouldn't damage your equipment due to the reconstruction filter.  At most it'll boost the volume a dB or two, probably not even for such mild clipping as you have in those images.


I find that slightly reassuring, but then why would the caskets (is that the right word?) of my headphones be hideously resonating, that is physically shaking on my head,  while the audio is distorting horribly?  This happened on my very old Sennheiser HD500s and on my brand new Sony ZX700s.  There was also unwanted vibration on my tiny, year old, in-ear Sennheiser CX95's, so it's hard to believe it's a construction issue.  This was a very unpleasant distortion with huge resonance.  The music is played on a ancient pipe organ, so the distortion is much more sustained than a quick peak.  It's the first and only piece of music I've encountered where Ogg Vorbis basically failed.  I guess it might be another killer sample?

Can lossy encoding create audible clipping not present in the source?

Reply #6
I downloaded the track and converted it into Ogg q5 and lame v2.  Interestingly, the lame sample does not clip, and I could not ABX it.

The vorbis sample does clip, but only very slightly.  With concentration after A/B'ing it for a while, I could hear a very slight hiss in the Ogg track that was not present in the mp3 or wav version during one of the organ blasts.  I'm assuming it was the clipping.  Once I locked onto that portion, I could ABX it 4/4 just by listening for the momentary slight bit of hiss during the otherwise fairly tonal organ blast. 

I then replaygained the sample, and as expected the bit of noise went away.  A very subtle effect, but with enough patience I believe audible.

Can lossy encoding create audible clipping not present in the source?

Reply #7
For a modern DAC, clipping shouldn't damage your equipment due to the reconstruction filter.  At most it'll boost the volume a dB or two, probably not even for such mild clipping as you have in those images.


I find that slightly reassuring, but then why would the caskets (is that the right word?) of my headphones be hideously resonating, that is physically shaking on my head,  while the audio is distorting horribly? 


That is definitely not clipping.  My guess is you have a buggy decoder on your system that overflowed during decoding.  I've fixed several bugs in decoders where you have a very loud, very pure tone and it causes integer overflow somewhere in the decoder.  This results in a very loud screeching sound as the flipped sign propagates through the codec.

Out of curiosity, what decoder were you using?

Can lossy encoding create audible clipping not present in the source?

Reply #8
..... but I see now that you saying that WavPack is the codec that is not doing this (obviously you are not using FFmpeg, or someone fixed that). However, it may that the decoder you are using for Ogg has a similar bug.


For the wavpack encode I used the wavpack command line tools from wavpack 4.6 in Debian.  Playback was via Rockbox 3.5.1 on iRiver H340, H140, Rockbox 3.10 on Sansa Clip+ and Rockbox daily build of 17 December on Android on an Archos A43IT (I wouldn't take too much notice of the Archos as a testing/comparison device because it seems to have a small built in bass and treble boost).

I didn't expect to ever hear a difference between high bitrate lossy and lossless,  and after doing abx tests I settled on using Vorbis -q 7 on the basis that it is overkill and I would never hear a difference...OK given some killer samples and a much lower bitrate I know I can abx lossy vs lossless but in normal listening it had been a very long time since something made me wince.

This sample got me frustrated and trying out different encoders and settings;  I was aware of wavpack but had hardly ever used it before.  I was surprised that it didn't affect the level like other (lossy) encoders, and that listening to the wavpack (without the .wvc, so lossy) on my Rockboxed devices the distortion simply wasn't there.  I didn't look for any other differences, I was just happy that my headphones didn't feel like they might jump off my head or fall apart.

Can lossy encoding create audible clipping not present in the source?

Reply #9
..... but I see now that you saying that WavPack is the codec that is not doing this (obviously you are not using FFmpeg, or someone fixed that). However, it may that the decoder you are using for Ogg has a similar bug.


For the wavpack encode I used the wavpack command line tools from wavpack 4.6 in Debian.  Playback was via Rockbox 3.5.1 on iRiver H340, H140, Rockbox 3.10 on Sansa Clip+ and Rockbox daily build of 17 December on Android on an Archos A43IT (I wouldn't take too much notice of the Archos as a testing/comparison device because it seems to have a small built in bass and treble boost).


If you have a sample that overflows the Rockbox Vorbis decoder, please post the exact file and a description of the exact player and version you ran it on so that it gets fixed. 

Can lossy encoding create audible clipping not present in the source?

Reply #10
I downloaded the track and converted it into Ogg q5 and lame v2.  Interestingly, the lame sample does not clip, and I could not ABX it.

The vorbis sample does clip, but only very slightly.  With concentration after A/B'ing it for a while, I could hear a very slight hiss in the Ogg track that was not present in the mp3 or wav version during one of the organ blasts.  I'm assuming it was the clipping.  Once I locked onto that portion, I could ABX it 4/4 just by listening for the momentary slight bit of hiss during the otherwise fairly tonal organ blast. 

I then replaygained the sample, and as expected the bit of noise went away.  A very subtle effect, but with enough patience I believe audible.


That's reassuring - it isn't just me 

I found that lame -h -V 1 was good for me.  Vorbis clipped at any setting, as did neroAacEnc whether one or two pass.  Once I stopped listening for the gross clipping I was able to notice some less obvious distortion in the few seconds leading up to it.....seems to be a sample with a variety of easily induceable artefacts...

Can lossy encoding create audible clipping not present in the source?

Reply #11
That's reassuring - it isn't just me 


I think the problem you reported is specific to you vorbis file.  Its probably some very unusual bug. Post the file so someone can look into fixing it.  What I'm reporting is just general clipping, which is a fact of life with any non-replaygained lossy file.

I found that lame -h -V 1 was good for me.  Vorbis clipped at any setting, as did neroAacEnc whether one or two pass.  Once I stopped listening for the gross clipping I was able to notice some less obvious distortion in the few seconds leading up to it.....seems to be a sample with a variety of easily induceable artefacts...


That won't actually prevent clipping.  Rockbox supports replaygain and a preamp, either will prevent this problem entirely.  Trying your WAV converted to Vorbis in Rockbox gave perfect output using replaygain.

Can lossy encoding create audible clipping not present in the source?

Reply #12
.... using replaygain.


I'm aware of RG and did use it but found it problematic with a collection containing lots of albums which RG gives a significant gain raise (big collection of early music i.e. lots of acoustic lute+voice and similar) as well as modern music that gets a substantial reduction.  I found that to guarantee avoiding clipping I needed a -6.0dB preamp, at which point it's no fun because everything except the loud stuff is much too quiet.  I settled on using the volume control instead.

Do you think the vorbis problem is with the encoder or the decoder?  Are you suggesting I post it here or to xiph or RB or somewhere else?



Can lossy encoding create audible clipping not present in the source?

Reply #13
I'm aware of RG and did use it but found it problematic with a collection containing lots of albums which RG gives a significant gain raise (big collection of early music i.e. lots of acoustic lute+voice and similar) as well as modern music that gets a substantial reduction.  I found that to guarantee avoiding clipping I needed a -6.0dB preamp, at which point it's no fun because everything except the loud stuff is much too quiet.


Its not necessary to use a preamp.  You can use the "prevent clipping" option. 

Although a -6dB preamp sounds really unreasonable to me.  Are you certain what you're hearing is actually clipping (and not some other problem as above)?

I settled on using the volume control instead.


Volume control just adjusts the amplification, so it won't help with clipping.

Do you think the vorbis problem is with the encoder or the decoder?  Are you suggesting I post it here or to xiph or RB or somewhere else?


I can't say what the problem is without access to the file that reproduces it.  You can post the file here and I'll handle it if its a rockbox bug.

 

Can lossy encoding create audible clipping not present in the source?

Reply #14
A few years back when I was using Winamp, Ogg Vorbis files would regularly clip with (near) 0dB material.

It turned out it was due to the Vorbis decoder, which had dithering enabled by default. Disabling dithering solved the clipping.

Can lossy encoding create audible clipping not present in the source?

Reply #15
A few years back when I was using Winamp, Ogg Vorbis files would regularly clip with (near) 0dB material.

It turned out it was due to the Vorbis decoder, which had dithering enabled by default. Disabling dithering solved the clipping.


I run Rockbox without dithering, but I don't know anything about how vorbis decodes.

It was well past 2:30AM here when Sartoga made his last post and I was getting more confused by my and Saratoga's differing experiences of apparently the same process, and too tired to think straight, so I slept on it (or tried to...huge loud storm all night) to approach it feeling a a bit fresher today.  I have started over with this rip+encode, the transcode, RG calculation, and playback to try to pin down what is happening, as there had to be more variables that matter than I'd initially considered.

Ripping: usually I'd use ripit, a perl script which calls cdparanoia, handles secure extraction (with suitable drive), naming, cddb lookup, tagging, RG calculation, playlist creation and so on.

This time I used EAC in wine.

I've made an accuraterip verified extraction and encoded to flac.  It gives me identical files (same checksum when decoded to pcm) as before except no RG data added by the ripping application, so now I move onto RG calculation and tagging and playback.

Rockbox default setting for RG is "Track Gain If Shuffling" which "reverts to album mode if SHUFFLE is set to No".  In practice I find that this enables RG in album mode if I load/listen to individual tracks, instead of my usual habit of loading/listening to a a complete album (directory or playlist). 

The RG metadata:

REPLAYGAIN_REFERENCE_LOUDNESS    : 89.0 dB

Using metaflac to add RG to the album gives me

Album replay gain                : -6.07 dB
Album replay gain peak          : 0.992035

and for track 03:

Replay gain                      : -6.51 dB
Replay gain peak                : 0.992035

There is some difference with the vorbis -q 7:

REPLAYGAIN_TRACK_PEAK=1.01847577
REPLAYGAIN_TRACK_GAIN=-6.55 dB
REPLAYGAIN_ALBUM_PEAK=1.02138662
REPLAYGAIN_ALBUM_GAIN=-6.10 dB

With wavpack, either hybrid or lossless, the results are close enough to identical to flac RG values.

So it's easy to see that the source audio has almost no headroom and that the Vorbis level is greater, and the RG tags reflect this.  But there is another factor, which is the way oggenc handles metadata when encoding from flac: it simply copies it, so it's possible to end up with a lossy track whose level has been raised to the point it measurably clips (if not audibly) but crucially it now has now inaccurate track and album peak and gain values which even make this slightly worse.....oh dear.

I've been using a script that takes any directory of audio files, mirrors the directory to output destination, transcodes to selected lossy codec, copies over *.jpg|*.bmp, runs vorbisgain/mp3gain/wvgain as appropriate and finally writes an extended .m3u playlist.  The vorbisgain part uses the -f (fast) option which means that no new calculation is made if RG tags already exist.  This meant that I got different RG tags when running the script on lossless files that hd no RG values than I originally got when running it on files that already had RG calculated.  I think it's also likely that I first noticed gross audible clipping when I listened to the track individually in Rockbox at default settings and didn't appreciate that this had triggered album mode RG with 0.0 preamp.

So what I've found today is that with RG disabled I don't hear the gross resonance that I heard before, but what does happen (not every time but when occuring it's always in the exact same place in the track) is a nasty click.  I wonder if this could be anything to do with the decoder overflow mentioned, or maybe is associated with the hardware.

Anyway my brain is about to implode now.

Can lossy encoding create audible clipping not present in the source?

Reply #16
I think it's also likely that I first noticed gross audible clipping when I listened to the track individually in Rockbox at default settings and didn't appreciate that this had triggered album mode RG with 0.0 preamp.


To be absolutely clear, whatever you're hearing in that track is not clipping.

So what I've found today is that with RG disabled I don't hear the gross resonance that I heard before, but what does happen (not every time but when occuring it's always in the exact same place in the track) is a nasty click.  I wonder if this could be anything to do with the decoder overflow mentioned, or maybe is associated with the hardware.


If you have a file that causes non-deterministic output in rockbox, you should post it now.

Can lossy encoding create audible clipping not present in the source?

Reply #17
].... a nasty click.  I wonder if this could be anything to do with the decoder overflow mentioned, or maybe is associated with the hardware.


If you have a file that causes non-deterministic output in rockbox, you should post it now.


I'm sorry I couldn't address this earlier.  Before I saw your reply I had connected my iRiver H340 to my PC and had set vorbsigain recursing through the H340 file system recalculating and applying the RG.  It took a while, and I didn't want to interrupt it and have to start over.

I have now pinpointed the remaining nasty click: 

Rockbox's RG setting is at "Track Gain If Shuffling".
I have been listening to an album and RG has not been activated.
Then I load a single track which has RG album value in the tags.  When RG is activated there is a often click as the gain changes.
The click occurs the after the same number of seconds each time it's heard, which makes it at first seem to be associated with the audio file, when in fact it's associated with the player.  Occasionally it doesn't appear at all, just to make things more frustrating and prove that this is the real world.....

The original gross distortion seems to have been caused by the way oggenc copies RG tags unchanged when encoding from flac, even though the levels of Ogg Vorbis file are not identical to that of the flac.  The same file with corrected RG values does not induce clipping.

So by forcing the recalculation/correction of the RG tags and/or by either disabling RG or forcing album mode I can play back the same track in the same codec with no audible clipping.

I can only thank you for your patience, and concede that I was mistaken in asserting that this particular artefact was only an effect of the lossy compression; in fact the elevated gain from the lossy compression was just one of several factors, and was probably the only one that might be considered obvious.  I am now going to lie down in darkened room and reminisce about when I was happy with a British Fidelity HF45 Record Player and a small collection of 45s and LPs, complete with buzzes, crackles, hums, the occasional smell of burning, and very little fidelity.

Can lossy encoding create audible clipping not present in the source?

Reply #18
Rockbox's RG setting is at "Track Gain If Shuffling".
I have been listening to an album and RG has not been activated.
Then I load a single track which has RG album value in the tags.  When RG is activated there is a often click as the gain changes.
The click occurs the after the same number of seconds each time it's heard, which makes it at first seem to be associated with the audio file, when in fact it's associated with the player.


I'm not sure I understand what you're saying.  Replaygain does not introduce a click.

The original gross distortion seems to have been caused by the way oggenc copies RG tags unchanged when encoding from flac, even though the levels of Ogg Vorbis file are not identical to that of the flac.  The same file with corrected RG values does not induce clipping.


I don't think this problem has anything to do with replaygain.  If processing with vorbisgain unbreaks the file, that is probably a coincidence. 

I can only thank you for your patience,


Great, but I am becoming increasingly impatient waiting for you to post a sample so that I can figure out what the actual problem is.

Can lossy encoding create audible clipping not present in the source?

Reply #19
I'm not sure I understand what you're saying.  Replaygain does not introduce a click.


Agreed: RG per se, with correct values, doesn't cause a click.  But the change in gain from one gain level to another can induce a click from the player, as does switching on eq or crossfeed, or the sleep timer shutting the player down and so on.  For example I have my Quick Screen set up so I can toggle EQ, or crossfeed.  If I toggle on or off EQ or crossfeed it isn't apparent absolutely instantly; it takes a few seconds (I suppose it takes effect when any already buffered audio is done?) and there is usually, but not invariably, an audible click as it does take effect.  That's what I heard, and it seemd to be associated with a particular time point in the playback when in fact it was wasn't.  So what I'm saying is that the encoded vorbis file is fine: that the original audible resonating distortion was an unforseen result of a combination of factors but I was mistakenly associating it with what was easily seen and overtly apparent - the thick red clipping warnings when I put the file into a wav editor, and the last artefact I noticed, a click, was being caused not by the audio file but by the player.

Can lossy encoding create audible clipping not present in the source?

Reply #20
I'm not sure I understand what you're saying.  Replaygain does not introduce a click.


Agreed: RG per se, with correct values, doesn't cause a click.  But the change in gain from one gain level to another can induce a click from the player, as does switching on eq or crossfeed, or the sleep timer shutting the player down and so on.


That is not how replaygain works.  Its a global adjustment to the entire volume of the track.  There is no click in the situation you described because the adjustment occurs prior to the start of playback.

So what I'm saying is that the encoded vorbis file is fine: that the original audible resonating distortion was an unforseen result of a combination of factors but I was mistakenly associating it with what was easily seen and overtly apparent - the thick red clipping warnings when I put the file into a wav editor, and the last artefact I noticed, a click, was being caused not by the audio file but by the player.


To be 100% clear:  there is no way an error of a fraction of a dB in the replaygain value could cause the integer wrap around you described above in an audio file.  That is simply not how these things work.

Now with that said, is there some particular reason you are unwilling to allow anyone else to determine what the actual problem is with your file?

Can lossy encoding create audible clipping not present in the source?

Reply #21
To be 100% clear:  there is no way an error of a fraction of a dB in the replaygain value could cause the integer wrap around you described above in an audio file.  That is simply not how these things work.
If something caused clipping prevention to kick in, then having the wrong peak value stored, and relying on it, will allow clipping - whereas the correct value would have enabled clipping prevention to work properly.

If the RG value used from the file is -6dB, then you'd need to use a pre-amp gain of +6dB or more to get clipping. With potential clipping (i.e. reconstructed sample values over 0dB FS) already in the file, a little less than +6dB pre-amp gain would bring these samples back above 0dB FS again, so giving clipping.


In short: copying over incorrect peak values is a problem when you rely on clipping prevention. However, with the pre-amp gain at 0dB and ReplayGain enabled, typical compressed pop music never triggers clipping prevention because it's already been turned down by 6dB (or more).

Cheers,
David.

Can lossy encoding create audible clipping not present in the source?

Reply #22
I downloaded the track and converted it into Ogg q5 and lame v2.  Interestingly, the lame sample does not clip, and I could not ABX it.

The vorbis sample does clip, but only very slightly.  With concentration after A/B'ing it for a while, I could hear a very slight hiss in the Ogg track that was not present in the mp3 or wav version during one of the organ blasts.  I'm assuming it was the clipping.  Once I locked onto that portion, I could ABX it 4/4 just by listening for the momentary slight bit of hiss during the otherwise fairly tonal organ blast. 

I then replaygained the sample, and as expected the bit of noise went away.  A very subtle effect, but with enough patience I believe audible.
I think Garf has been looking for a positive ABX of audible clipping due to lossy encoding like this for years. This certainly points the finger in the right direction. Many thanks.

Cheers,
David.