Hi community,
I feel like I've searched the whole internet, but still could not find any satisfying answer to my question.
Currently, my music collection is encoded in LAME MP3, but I would like to switch over to Ogg Vorbis, at least for those albums I have archived as FLAC. I expect the same quality (transparent, which LAME was at V2, V1 and V0 for me - used all of that switches all across my collection in the last few years) for lower bitrate. Still have to ABX-Test, which quality setting of Ogg Vorbis will be transparent to my ears, but well, let's go over to my actual question.
In LAME, there is the --nogap switch, which ensures that consecutive tracks are encoded gaplessly. After that, we have to give Lame all the tracks to encode at once like:
--nogap <file1> <file2> <...>
As stated in the reference.
No, I know that Ogg Vorbis is gapless by design, but I wonder, if there is anything I need to be aware of when transcoding my FLAC-Collection to Ogg Vorbis, anything similar to the --nogap switch of LAME? I wonder, if I just can encode track by track independently, even ignoring the track order, in a batch file, and assume that tracks that are meant to be gapless, are encoded correctly at their transitions?
The Vorbis tracks will be as gapless as whatever you're transcoding from (apart from tiny lossy artefacts at the frame boundary). Assuming the Flac is gapless (usually is) then the Vorbis will be too. This should be true whichever encoder you use and whichever order you process the tracks.
Do you actually need the nogap hack in lame? If you are thinking about vorbis I'd be surprised if you had software old enough to need it.
The Vorbis tracks will be as gapless as whatever you're transcoding from (apart from tiny lossy artefacts at the frame boundary). Assuming the Flac is gapless (usually is) then the Vorbis will be too. This should be true whichever encoder you use and whichever order you process the tracks.
Thanks :)
Do you actually need the nogap hack in lame? If you are thinking about vorbis I'd be surprised if you had software old enough to need it.
Well, I actually don't know
if I really need it. When I originally ripped my discs using EAC, I did not engage that --nogap switch.
Normally I used Poweramp for listening on my Android, and VLC on my computer, until I found foobar2000 which fully replaced VLC for music-listening-purposes.
I don't really know, how good different software deals with gapless playback, even if tracks might not be encoded properly at their transitions. Just found that Poweramp and foobar2000 always play it nice and gapless.
Recently, I played around with Phonograph on android, which I actually really liked because of its playlist features. The bad thing about Phonograph is, that is uses the inbuilt Android Audio API, which in my experience can be a mess. This is the only application where I noticed that some of my LAME MP3s which are meant to be played gaplessly, actually have a short glitch on track transitions. Gapless playback with Ogg Vorbis and FLAC does not work here at all, and as I did some testing, I found that it always worked for me, if I used the --nogap switch of LAME. For LAME MP3s, that were not encoded using --nogap, it actually worked
sometimes.
Nevertheless, I'm gonna stay with poweramp on Android because of its much better sound-features like the equalizer, bass- and treble-adjustment, stereo enhancer etc... So, I don't care if something like Phonograph is not able to play Ogg Vorbis or even FLAC gaplessly, because Poweramp does.
Still I wanted to ask the question I opened this thread with, because I am a little perfectionist and I wanted to make sure that my Ogg Vorbis encodings support even players, where gapless playback is poorly implemented, so you could eventually hear glitches if files are not properly encoded. We never know which player we will use in the future! :)
So I think I'm going fine with just the default settings of Vorbis, encoding track by track and automatically achieving the best possible gapless playable tracks in the end.
Thank you! :)
In that case you probably do not want the nogap switch. It's meant for very old hardware that could do wav gapless but did not support gapless mp3.
It's not worth explaining --nogap. Just don't use it. It is an abandoned feature which requires special player support.
Oh okay, good to know. I thought it ensures that the audio stream in the MP3 file contains the necessary information at track boundaries to make it really gapless playable.
No, LAME adds the necessary info (which still requires special support) to the files it creates normally. --nogap was an earlier and somewhat different method. Neither is perfect.
Do you have any reason to believe gapless decoding of individual tracks created with Lame is any less perfect than what is done with other lossy formats, including that found in the thread title?
I have no experience with the way other lossy formats do it, but the way it was explained for LAME was that encoding each file separately means the psychoacoustic model (e.g., what frequencies are masked/discardable) is reset for each file. The resulting waveform might have a jump in it when the two files are played together. I've seen this in my own testing. Sometimes the jump is enough to be audible as a click, which is not a gap, per se, though the effect is similar. One of the developers said there is a theoretical risk that quality could momentarily drop as well (e.g. due to resetting the bit reservoir) but there are no examples where this has actually been audible.
It seems unlikely other lossy formats would be any better in this regard.
theoretical risk that quality could momentarily drop as well (e.g. due to resetting the bit reservoir)
I'll have to admit I'm quite ignorant of how this can be. If it is true that a capable player simply discards X samples at the beginning and Y samples at the end during the decoding process and prior to buffering, I don't see the difference making use of gapless information would have. Would this concern also be applicable to the beginning and/or end of mp3 files in general?
the bit reservoir thing does not sound plausible.
The main problem I am aware of is that there is no continuity from one file to the next since the first encoder instance does not know what the first sample of the second will be. In theory there could be some minor difference in amplitude if the last sample of the first track quantized down while the first of the second is quantized up.
The main problem I am aware of is that there is no continuity from one file to the next since the first encoder instance does not know what the first sample of the second will be. In theory there could be some minor difference in amplitude if the last sample of the first track quantized down while the first of the second is quantized up.
Yes, and that cannot be unique to the mp3 format.
Sorry, "bit reservoir" was my own speculation at to what Gabriel was referring to when he said this:
My personnal advice would be to not waste time with the --nogap option.
It is a hack, and can only work with cooperation of the player. The
problem is that when you look at the individual files it produces, they
are a little "unusual", with potential quality decrease at the beginning
and at the end.
Ref: https://sourceforge.net/p/lame/mailman/message/9923039/
I tried to ask for details on lame-dev a while back, but didn't get anywhere: https://sourceforge.net/p/lame/mailman/message/27315501/
The problem I found with --nogap or otherwise-naïvely-split files is that in a non-supporting player like foobar2000, the end of the file is never reached; the last however-many samples (the same number as the decoder delay) won't be obtained from the decoder unless another frame is fed in (preferably the first frame of the next file to be played, although silence might work). So when playing each file individually, for all but the last file in a set, you'll be missing some samples at the end of each track. I don't think this is what Gabriel was talking about when he mentioned quality, though.
You said neither method was perfect. I thought it was fairly obvious that I was talking about Lame's current and long-standing implementation of providing metadata necessary for gapless playback.
There is no point discussing an option that never got off the ground and was subsequently replaced well over a decade ago.
Not sure why you think we recommended --nogap for rockbox. Our wiki entry on gapless (last updated 2007) is pretty clear about that this is not necessary except on the old players that have a hardware decoder chip that cannot do gapless playback, and only then it is one of several hacks you can try to workaround lack of hardware support:
http://www.rockbox.org/wiki/GaplessHowTo
Greynol, when I said "One of the developers said there is a theoretical risk that quality could momentarily drop" I was half-remembering what Gabriel said on lame-dev. He did indeed say that, but the part I didn't remember correctly was that it was in reference to the playback of files created with --nogap. I was thinking it was in reference to the encoding of separate files (the normal way), which would make more sense since there was already the issue of not having access to adjacent tracks.
If people have misconceptions about what --nogap does and whether it's needed/useful, then I am going to try to correct them. You can say there's "no point" to that, but we are on the same side, here. Ostensibly.
I was thinking it was in reference to the encoding of separate files (the normal way), which would make more sense since there was already the issue of not having access to adjacent tracks.
My point is that by not attributing this problem to lossy formats in general, it appears as if you are talking about it as if it were not just mp3-specific, but lame-specific (ignoring that Apple has a gapless implementation of mp3 as well, if not others).
(the normal way)
A key point to this discussion is that the OP didn't (still doesn't?) know what this means.
If people have misconceptions about what --nogap does and whether it's needed/useful, then I am going to try to correct them. You can say there's "no point" to that
There is a very valid point to that, but only so long as it is clear that this is not valid as it relates to what is being done today. Unfortunately muddying the waters with half-remembrances isn't helping matters, especially when presented in more detail than is necessary and as if it were factual. It ends up obfuscating the truth:
Lame's default built-in implementation of providing information so a player can skip extraneous samples which has been in place for well over a decade is no poorer than that of any other lossy codec that creates extraneous samples upon decoding.
There were no half-remembrances until after you pounced on my statement "Neither is perfect," which is factual.
Instead of simply saying "Lame's default built-in implementation of providing information so a player can skip extraneous samples which has been in place for well over a decade is no poorer than that of any other lossy codec that creates extraneous samples upon decoding," you instead asked "Do you have any reason to believe gapless decoding of individual tracks created with Lame is any less perfect than what is done with other lossy formats, including that found in the thread title?"
Although I should know better than to respond to your rhetorical questions, I think my reply was concise and accurate (aside from what should have been an inconsequential misrepresentation of Gabriel's lame-dev comment), and it should have completely satisfied you and the readers who you assume were completely confused by everything I say.
I would have left it as an exercise for the reader to research why neither method is perfect, but rather (at best) just-good-enough. All that needed to be said was said in the first 9 replies.
Sigh.
All that needed to be said was said in the first 9 replies.
Except that you completely overstepped in post #9 with contextually misleading* and possibly untrue** information:
"One of the developers said there is a theoretical risk that quality could momentarily drop as well (e.g. due to resetting the bit reservoir) but there are no examples where this has actually been audible."
(*) how was anyone to know that you were talking about --nogap in a paragraph that had, until then, been devoted to the current method of adding metadata?!?
(**) what is bold in the quote above.
Had I been satisfied that the OP would understand that abnormal amplitude discontinuities at track boundaries was a lossy problem in general I wouldn't have made anything of it. I won't lie: I found your subsequent decision to flail over --nogap (as if it were still on the table, which it wasn't) annoying. I do, however, feel bad about the apparent misunderstanding.
I think expecting 100% transparent gapless from MP3 overlooks the fact that MP3 is a lossy format that cannot 100% ensure transparency. All LAME can do is make sure that your input file has the correct number of samples, not that the quantization error at the track transition will be masked.
This isn't a problem with gapless though, its just how lossy audio works.
Description
The --nogap option is as if the encoder was fed a continuous wav file but produced separated mp3 files. It is conceptually similar to having an mp3 file and split it with adequate tools (with some care about not using bit-reservour for a frame on the next file).
This removes the encoder delay (all but the first file) and padding (all but last file). A player that doesn't know about gapless metadata, but is able to decode two consecutive files without adding any type of pause, is able to remove decoder delay, and also does not remove a default encoder delay, could play these files near gapless, whereas they would have notable gaps otherwise.
Test case
I've tried encoding two samples with:
lame -V0 file1.wav file1-metadata.mp3
lame -V0 file2.wav file2-metadata.mp3
lame -V0 --nogaptags --nogap file1.wav file2.wav
lame -V0 --nogap file1.wav file2.wav
and this is the outcome:
As it is to be expected, foobar2000 reports that the two files encoded with gapless info decode to the exact same amount of samples than the original wav files.
As is to be expected, foobar2000 reports that the first file encoded with nogap has a encoder padding of 0. The second file (the last) has a positive encoder padding.
As is to be expected too, the individual amount of samples of the "nogaptags" files is not equal to the amount of samples of the individual wav files. (it has to cut at mp3 frame boundaries).
It is not expected that the total sample size for the "nogaptags" files is smaller than total size of samples of the original wavs. (since it has gapless metadata one would expect that the total decode size is the same).
I decoded the "nogapstag" with both, foobar200 and lame, and they both generated the same amount of samples. I compared them in audacity and the second file misses samples on the beginning. The end is at the correct position on both files.
I decoded the "nogap" counterpart too. It has an additional encoder delay at the beginning (as is to be expected), but the second file's start is also cut.
I redid the test with CBR (-b128) instead of VBR and resampled wavs to 44Khz (initial test was 48Khz). I got very similar results.
Conclusions
As it is now, the --nogap option is counterproductive. It is not guaranteed that it is useful on any current player, and it is guaranteed to be worse on decoders that can do gapless decoding.
Ogg Vorbis and similar lossy codecs that inherently support gapless playback still suffer from theoretical lossy artefacts at the track boundaries. I can't claim to have ever detected that with my ears. The magnitude of such artefacts would depend on the importance of the cross-lapping (or lack of it) at frame boundaries in a particular codec. I have no clue if such things would be more or less serious in MP3 than other codecs, but I'm guessing they are effectively inaudible in any codec. The real problem is getting the timings right, once that is done it sounds 100% gapless.
So, after all, summarizing the posts, a short answer to my initial question could be:
No matter if I use, Vorbis, Lame or any other lossy encoder, I am best off, if I just encode each file individually, and just don't worry about the track transitions. The encoders do all they can, and if I use gapless-capable players like Poweramp on Android or foobar2000 on Windows (or also Android), everything will just be fine if I encode each file seperately.
Correct?
Since you mention foobar2000, why not use that to transcode your files? It has presets for LAME and vorbis and all you need to do is set the quality setting via a slider. You don't have to worry about any command line settings. The defaults have always resulted in perfect gapless files for me.
The first time you use it, you'll need to point it at the command line encoder like lame.exe, oggenc2.exe, etc.
http://wiki.hydrogenaud.io/index.php?title=Foobar2000:Converter
Yep, I know that I could also use foobar2000 for transcoding, but as I am a programmer, I like to have the full control with scripts anyway, so I prefer having my own batch file that fires the encoding processes.
I'd like to know and understand any detail that is involved, as much as possible. That's also why I asked that question in that thread here. I know, with the correct players my music already worked gapless before, but I wanted to look behind and understand what's going on, and if I can even improve.
Well I guess, foobar2000 will do nothing more than also just the default command line settings of the encoders, right? Is there any way to prove that, like some log file where I can see the commands used to encode the files?
Nevertheless, I prefer having my batch file, that I can adjust completely free and just double click to let transcoding begin :)
Okay, I found out, if I select an encoder from the foobar2000 convert-dialog, and then select "Custom", the fields are filled with the command line switches obviously used for the encoder selected before. So I could see that foobar just uses the Lame Default settings with the selected quality setting plus the --noreplaygain switch. Which is actually just fine for me, because I don't use replaygain at all, and if I would, I would not use it based on the single track gain, because normally I listen to complete albums.
Well, I encoded two albums at once to MP3 using foobar.
After that I wrote a batch file where I encoded track by track with single calls of Lame, same quality setting (V0) as used in foobar2000 and of course also the --noreplaygain switch.
I wanted to see, if there are any differences in the results, maybe foobar2000 could feed the encoder with one big stream of the whole album, to improve gapless playback. If this would have been the case, the resulting files would be different from those encoded individually from cmd.
But there are not any differences. I used WinMerge to compare the encoded files of foobar2000 with those from cmd, and it tells me that all files are binary identical.
I see this as a proof, that foobar2000 actually does the same like when I encode each file individually via command line.
I still find it sort of cool, running my batch file and let it encode all files in the console, but I think in this case, foobar2000 is the winner of the match, because it can automatically transfer the metadata to the encoded files when finished encoding.
So I think, I'm gonna use foobar2000 for my transcoding, in contrast to what I said first, that I would prefer having my batch file running.
Wanted to test the same thing with Ogg Vorbis. There is that -s command line switch in oggenc2, that seems to be fed with a random number by foobar2000. The internet says, this switch forces including a serial number into the output stream, and this is primary used for testing. I don't have a clue why foobar2000 uses that switch.
Any idea? I think this switch is the reason why I can't get binary identical output files when encoding vorbis from cmd. Because of the random number. What's the exact point of that -s switch?
Ogg chaining requires each chained file to have unique serial number.
So, the other way around, what would be the problem if to consecutive Ogg Vorbis files had the same serial number? I don't really understand what that serial number is used for.
A program opening the file would not know about its special nature without fully decoding it. It depends on programming what would happen but it could show wrong file duration and playback could end at the end of the first stream.
So this is actually more than just a feature for testing, like i read somewhere else?
Next question: What is a Vorbis stream? Or would the word "Ogg-Stream" be correct here? I don't really have any deeper knowledge about ogg and vorbis. Does an ogg Vorbis file consist of many streams that are played after each other? Or is a stream just the actual audio data of the file (so, without metadata and so on).
I think lack of a unique serial number would mean that you couldn't later append the ogg files into a single file without editing the number.
In practice I don't know if it matters. Chaining files is rarely used and a lot of software doesn't support it anyway.
An Ogg file can optionally consist of multiple streams literally just concatenated together. Streams may also be multiplexed so, for example, a video and an audio stream can have pages interlaced. The serial number is obviously important in those cases. Concatenated streams in a single file are referred to as chained. You're most likely to see this in an internet stream where they just throw one track after another at you in a continuous flow of bytes.
oggenc should generate unique serial numbers all on its own, whether for different running instances or for multiple streams chained within the same file. Perhaps it didn't do this at some point, or perhaps foobar is just being paranoid. Maybe it is something specific to the way foobar encodes?
Having the same serial number for multiple streams within the same file can cause problems for decoders in some circumstances. You may see things like incorrect stream lengths reported, or seeking may get borked. It isn't common, but then it shouldn't happen because the serial number is supposed to be unique. I've run into it by creating test chained files using streams with the same serial number, but I can't remember exactly the things that broke.
So what an encoder for gapless "should" do (not that I claim failing to do so will produce annoying artifacts), is
- know what track selection constitutes an "album"
- encode each album as single track
- split into individual files
?
The next question is: what piece of software will do so?
If none do in one operation: is it possible to encode to image-with-cuesheet using one application, and automatically split by cuesheet using another, bringing it down to two batch operations?
So what an encoder for gapless "should" do (not that I claim failing to do so will produce annoying artifacts), is
- know what track selection constitutes an "album"
- encode each album as single track
- split into individual files
?
The next question is: what piece of software will do so?
If none do in one operation: is it possible to encode to image-with-cuesheet using one application, and automatically split by cuesheet using another, bringing it down to two batch operations?
I think you're overthinking this :) A lossless codec knows exactly where the start and end of each track is, down to the exact sample (implicitly, it doesn't require any special mechanism). Transcoding to a lossy format simply requires a mechanism to identify the exact starting and ending sample, or time, so that a decoder can play the first sample of one track seamlessly following from the last sample of the previous track. If you have a single lossless file plus cuesheet, nothing has really changed so long as the lossless decoder understands where the track boundaries are.
Ogg Vorbis has that built in, you don't need to do anything special, you don't need to encode in any special order or known anything about albums. MP3 doesn't have that built into the codec definition, but Lame has come up with a mechanism that is widely supported. AAC also doesn't include such information, and more than one method of storing track boundaries in metedata is in use, which leads to difficulties when the encoder and decoder don't use the same scheme.
The old MP3 -nogap option did something different because it (or at least decoders) didn't have the means to identify the track boundaries properly. It actually overlapped the track starts and ends, which obviously required tracks to be transcoded in order and could cause artefacts if the tracks were played individually. Your three steps of what a gapless encode "should" do describes this, but it isn't the way any "modern" encoder works. So, to answer your last question, the only encoder I know of that will do these things is Lame with -nogap, but since it is an obsolete approach the point is perhaps moot.
I think expecting 100% transparent gapless from MP3 overlooks the fact that MP3 is a lossy format that cannot 100% ensure transparency. All LAME can do is make sure that your input file has the correct number of samples, not that the quantization error at the track transition will be masked.
This isn't a problem with gapless though, its just how lossy audio works.
This.
Now to meander a little beyond...
A little research should provide a massive amount of information (this has been discussed heavily, in fact it is the topic that brought me to this forum some 15 years ago). Anyway, here are a couple topics I pulled more or less at random to get people thinking:
https://hydrogenaud.io/index.php/topic,18211.msg179288.html
https://hydrogenaud.io/index.php/topic,35654.0.html
And from the horse's mouth:
https://xiph.org/vorbis/doc/vorbisfile/crosslap.html