HydrogenAudio

Lossless Audio Compression => FLAC => Topic started by: madah on 2002-02-13 16:21:42

Title: Ogg+Flac as lossy+recovery file!
Post by: madah on 2002-02-13 16:21:42
Here's a quick test I did:

I encoded a wav-file (Eurythmics' Sweet Dreams) to ogg using Oggenc RC3 -q 5.0, then decoded it to wav. The decoded Ogg was then inverse-mixed (using cool edit) with the original wav file, then compressed with Flac.

Original WAV size: 49.4 MB
Original FLAC compressed: 28.3 MB

Ogg: 5.56 MB
Flac 'Recovery' file: 24.8 MB
Total (ogg+recovery): 30.4 MB

As you can see, the Ogg+Recovery solution is only 7% bigger than the original flac.

To reverse the process (ie create the original wav): just mix the decoded Ogg and the decoded Flac-recovery-file togheter!

Since both Ogg and Flac are open-source, this could easily be automated.

I know that WavPack and Monkey's Audio have this lossy+recovery, but they are closed-source windows-only.

So what do everybody think about this?
Title: Ogg+Flac as lossy+recovery file!
Post by: CiTay on 2002-02-13 17:01:30
I don't get the point of this. FLAC is already lossless. You'll get the original WAV when you decode from FLAC.

And what makes you so sure that a lossless inverse-mix and a lossy Ogg will amount to the original WAV file? I don't think this works.
Title: Ogg+Flac as lossy+recovery file!
Post by: JohnV on 2002-02-13 17:42:54
Umm difference signal and decoded ogg mixed together should amount the original.

The idea is that you can first download the ogg, if you later want to make it lossless you'll save few MBs worth of downloading by downloading the recovery file instead of original lossless.

Also I don't know if RIAA can attack you for hosting only difference signal..

Not sure how useful this would be though.
Title: Ogg+Flac as lossy+recovery file!
Post by: CiTay on 2002-02-13 17:57:46
Quote
Originally posted by JohnV
Umm difference signal and decoded ogg mixed together should amount the original.


Do you really think that the mixing, the decoding and the additional mixing are accurate enough to give you a WAV with the same CRC as the original? I have to see proof before i believe that.

Quote
The idea is that you can first download the ogg, if you later want to make it lossless you'll save few MBs worth of downloading by downloading the recovery file instead of original lossless.


Okay, but you could as well download the file off WinMX in 128 kbit MP3 'quality' first, listen to it, and if you like it, download the full FLAC (from whereever) and delete the MP3. Cause if you don't like the downloaded OGG, why would you wanna keep it?
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-13 19:04:46
I think this is an awesome development if in fact it would result in identically CRCd files.... when WAVpack came out with this feature a few weeks ago, I tried it out and thought it was an awesome idea, but wished it could be implemented in a codec better tuned for lossy compression  - wavpack's lossy 320 kbps, although I didn't do extensive listening tests, I would imagine would be nowhere near the quality of ogg, mpc, or even mp3, as it has not undergone the extensive tuning those formats have.

For those who don't see the point of this, the reason I think its useful is because I want to encode my entire CD collection, but the current state of audiocoding is in such flux right now, I am having a hard time deciding on a format... do I go with MPC for quality, but then have to transcode to mp3 for hardware players? Do I go with ogg and hope that hardware player support that has been talked about actually comes through? If so, do I go ahead and encode now, or wait for 1.0? A feature such as this solves this problem, as I could encode in ogg for now, burn all the correction files to backups, so I have something to listen to for now, without requiring the massive disk space that would be required to store them all lossless on my HD... but then when the state of audiocoding settles down and I decide on a final format, I can restore the original waves and reencode without worrying about quality loss from transcoding.
Title: Ogg+Flac as lossy+recovery file!
Post by: Tom Servo on 2002-02-13 19:19:03
Lossless coders a la FLAC, LPAC and Monkeys Audio aren't really suited for compressing a recovery file.

If you inverse-mix the OGG decoded and original, it'll give you mostly fuzz, which these lossless coders really dont handle good. If there would be some lossless encoder dedicated to compress these fuzz files, you'd achieve a higher ratio.
Title: Ogg+Flac as lossy+recovery file!
Post by: CiTay on 2002-02-13 19:30:47
I still have a hard time understanding the philosophy behind this all, Randum.

Are you telling me that you prefer a backup with two seperate files, of which a) the Ogg file will possibly become worthless when you settle on a different format/version in the future, b) the FLAC file _alone_ is already worthless, and c) where you can't re-encode without taking both files and joining them first...  over a backup with one lossless FLAC copy and one small file for your hard disk, encoded with what you like best at this very moment, and where you can easily re-encode from the FLAC files off your CDs, when - for instance - Ogg Vorbis 1.0 is available?
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-13 19:54:31
Well, the more I think about it, the less convinced I am that madah's proposed method would actually result in bit-identical original wav's... however, if it could be made to work, I still think it would be very useful. CiTay, your suggestion that for my purposes I should just encode 2 versions, a lossy and a lossless, and burn the lossless to backup... that would, of course, work for my purposes, it would just bug me that I'm using backup media unnecesssarily for redundant data. With wavpack, the correction files are significantly smaller than the entire lossless file, so it would increase the number of backups I could fit on a CDR (or whatever backup medium I use)... if the same could be said of the hybrid ogg method (or... if it does work, it could be equally applicable to mpc) you would have the same benfit, just with better sounding lossy files. Essentially the only reason to do your way over mine/madah's would be all the additional steps required for the process of generating and compressing the correction files... which is of course a totally valid reason - I would never consider using this method unless someone (me?) took the time to write an automated tool to do this kind of thing in a simple one step frontend.
Title: Ogg+Flac as lossy+recovery file!
Post by: JohnV on 2002-02-13 20:08:37
Quote
Originally posted by CiTay
Do you really think that the mixing, the decoding and the additional mixing are accurate enough to give you a WAV with the same CRC as the original? I have to see proof before i believe that.
Sure, you can verify this easily with CoolEdit.
Title: Ogg+Flac as lossy+recovery file!
Post by: Gecko on 2002-02-13 22:31:07
Now I get it!

Create lossy file - Create difference file lossy/original - Compress difference file using a lossless scheme.
With this it is possible to get a bit accurate reproduction of the original. Interesting idea yet I do not see the benefits.

The size difference of the two lossless files in this case is only little more than 12%.  If you want to backup for later reencode then why not just "waste" those 12% and have files you can reuse? Imagine: you decide to settle on ogg 1.0 but maybe someday you will want to reencode that also. So you need to create another set of recovery files etc. Even if automated this is still time and media consuming. Or another one: you loose your lossy files (hdd crash, accidental deletion...) and think: if only I had not saved those 12% I could still listen to the music today.

Perhaps you will be better off using general purpose compression since, as Tom pointed out, you are not necessarily encoding tonal data.
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-13 22:49:59
Quote
Originally posted by Gecko
Or another one: you loose your lossy files (hdd crash, accidental deletion...) and think: if only I had not saved those 12% I could still listen to the music today.

Perhaps you will be better off using general purpose compression since, as Tom pointed out, you are not necessarily encoding tonal data.


Yes, I thought of this too, and I think its the most convincing argument against this method - and why I'm probably not going to actually do it. Still, I just think the idea has a coolness factor to it
Title: Ogg+Flac as lossy+recovery file!
Post by: madah on 2002-02-13 23:36:40
Quote
And what makes you so sure that a lossless inverse-mix and a lossy Ogg will amount to the original WAV file? I don't think this works.


I've tested the files and they where bit-identical. But, if the decoder is changed or different decoding method is used (replaygain/dither/etc...) this will of course fail...

Quote
The idea is that you can first download the ogg, if you later want to make it lossless you'll save few MBs worth of downloading by downloading the recovery file instead of original lossless.


One thing that this could be very useful for is that you can download a preview mp3/ogg for free (from an artist' page) and then pay to get the recovery file if you want full quality...
Title: Ogg+Flac as lossy+recovery file!
Post by: bryant on 2002-02-13 23:54:45
I think that FLAC (and other lossless compressors) would probably do pretty well on the difference files. They are still 16-bit stereo audio and all those programs are adaptive to some extent, although a little alternate tuning might help. I'm sure they would still compress much better than a general purpose compressor.

However, if someone wants to use this method, they should store the decoder executable along with the ogg files. Different versions of the decoder would very likely produce slightly different decoded versions (just like different MP3 decoders). While these differences would probably not be audible, they would cause the "recovery" file to no longer work exactly the same (i.e. bye-bye lossless).

madah: Wrote this before your last post showed up!
Title: Ogg+Flac as lossy+recovery file!
Post by: CiTay on 2002-02-14 00:43:32
Quote
Originally posted by madah

One thing that this could be very useful for is that you can download a preview mp3/ogg for free (from an artist' page) and then pay to get the recovery file if you want full quality...


If bandwidth is really the issue here, why not pay for the CD, instead of downloading an additional recovery file...

I can think of no situation that would make the space saving worth the extra effort and doubled risk of data loss.
Title: Ogg+Flac as lossy+recovery file!
Post by: mithrandir on 2002-02-14 01:29:53
:confused:

I fail to see the value in this.

There are two situations: a) I want to encode an album where I have the original CD. b) I want to encode an album where I will not have the original CD in the future.

For case a), there is no need to create a recovery file and burn it to disk. Just re-rip the original CD and re-encode.

For case b), I pick a lossy format that offers significant headroom and just bite the bullet and realize it will be a permanently lossy copy. For these situations, I currently use "mppenc --xtreme --nmt 16 --tmn 32". Avg bitrate ranges from 275-325kbps...a filesize that I can manage with current disk space realities. It will be extremely rare if I can tell these MPCs apart from the WAVs.

For case b), if in the future I realize that I shouldn't have used a lossy format and wished I had the original WAVs again to re-encode, I could always buy the CD. If this CD was known to be OOP/rare/etc, I would have used a lossless format like LPAC or MAC.
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-14 02:52:09
Quote
Originally posted by mithrandir
I fail to see the value in this.

You're neglecting a couple of scenarios. The whole point of this is to not HAVE to 'bite the bullet'.... as stated, the reason I want to do this is so that I can have small file sizes on my hard drive, yet still be able to reencode them from the original wav's at a later time should either a) a new audio codec comes out that I like better, or b) I hear artifacts in my lossy versions, and decide that I should have encoded at a higher bitrate.

Of course, there are other reasons which have been pointed out not to do it this way, most notably the fact that just storing complete, lossless versions rather than correction files would take only a bit more space, and serve the dual purpose of avoiding 'bullet-biting'  and serve as an actual backup should you have a drive crash (or brain fart) and lose your lossy versions. This, however, is the rationale for wanting this kind of functionality in the first place.

Edit: I kant spel gud
Title: Ogg+Flac as lossy+recovery file!
Post by: layer3maniac on 2002-02-14 03:19:57
What is the point? Isn't that a lot of trouble to save 3 megabytes?
Title: Ogg+Flac as lossy+recovery file!
Post by: mithrandir on 2002-02-14 04:18:56
Quote
The whole point of this is to not HAVE to 'bite the bullet'.

Sometimes we need to take a step back and ask "what exactly are we doing?"

There are two things we must consider: diminishing returns and opportunity costs.

Tonight I encoded Live's A Distance To Here album in MAC's Normal mode and the average bitrate came to 988kbps. This is simply unacceptable compression from a practicality standpoint; not that it's MAC's fault, but lossless compression does not pay for its benefits, IMO. The same album encoded with my MPC settings required 284kbps, less than 1/3 of the lossless size. I simply cannot tell the MPC files from the originals. It's more than "good enough"...my ears think they are flawless.

If my original album were destroyed and I could never, ever get another original copy, I probably wouldn't "care" from an audio standpoint. What am I losing? Some wispy fine detail -60dB down in level that my ears can't hear anyway? Oh well, life goes on.

I personally believe you can get "perfect sound" from existing lossy codecs. It's all a matter of bitrate. Can any human tell the difference between a 450kbps MPC/Ogg file and a 900kbps lossless? I don't think so. That's diminishing returns.

We also have time limitations. Time IS money, to use a oft-used phrase. To go through this effort of making two files with two separate encoders, then burning one of them to media that can only handle 700MB (just a few albums worth)...forget it. In 10-15 years, I will probably rebuy my albums all over again in 24-bit, 96KHz format, so really, I think you can really go overboard with perfectionist tendencies. I found an encoder and a command switch combo that I'm very happy with and I am sticking to it.

From an academic standpoint, this lossy+FLAC exercise might be interesting, but it's not practical nor particularly necessary for "normal listening" needs.
Title: Ogg+Flac as lossy+recovery file!
Post by: kritip on 2002-02-14 11:39:17
One more thing, correct me if i have got this idea all wrong, but you'd store the 'lossless' file on CD for example and the ogg's on your harddrive and combine them again to create the original wav so you save a few megs on the media. What happens if you lose all your ogg's (recovery files) in the event of a hard disk faliure, won't the archived files then be of no use because they can't be recovered??

Sorry if i got this idea all wrong, seems i may have done since this hasn't been mentioned yet, but if it is correct then it's an important factor, you would infact need to bck up the ogg onto the CD or other medium as well!

Kris
Title: Ogg+Flac as lossy+recovery file!
Post by: Timmy The Turtle on 2002-02-14 13:35:19
Just a quick question to mithrandir. Is your modified mpc command line "mppenc --xtreme --nmt 16 --tmn 32" better than the standard --insane ? And if so why didn't you use a modified version of insane. I ask this because your average bitrate seemed to be a lot higher than I get with insane and 0.90s.

Thanks:)
Title: Ogg+Flac as lossy+recovery file!
Post by: madah on 2002-02-14 13:56:16
Quote
What happens if you lose all your ogg's (recovery files) in the event of a hard disk faliure, won't the archived files then be of no use because they can't be recovered??


Yes you're right. I personally would never use this for backup purposes, unless both the ogg and flac are stored. But storing the original flac alone will result in less space so this is useless...

I do find the idea of a lossy+recovery combo very interesting though...

It would better be used for other purposes, for example: You have a link to an insane 1400 kbps Ogg (let's say it's lossless). You download only so you get 128 kbps bitrate. Later you find you'd like to have better quality, so you download a difference-stream of 64 kbps, so total bitrate of the file will be 192 kbps. And if you want full quality, you only have to download 1208 kbps (1400-192) compared to the full file.

Maybe Ogg bitrate-coupling will work this way, or is it only capable of scaling the bitrate down?
Title: Ogg+Flac as lossy+recovery file!
Post by: Benjamin Lebsanft on 2002-02-14 14:20:57
Quote
Originally posted by madah
Maybe Ogg bitrate-coupling will work this way, or is it only capable of scaling the bitrate down?


i think it is called bitrate peeling and it would be really nice if it could work the way you described it
Title: Ogg+Flac as lossy+recovery file!
Post by: mithrandir on 2002-02-14 17:08:14
Quote
Originally posted by Timmy The Turtle
Is your modified mpc command line "mppenc --xtreme --nmt 16 --tmn 32" better than the standard --insane ? And if so why didn't you use a modified version of insane. I ask this because your average bitrate seemed to be a lot higher than I get with insane and 0.90s.

As I learned from Dibrom in another thread, --xtreme sounds better than --insane at equal bitrates. This is because --xtreme uses the ATH whereas --insane encodes the full 22.05KHz bandwidth, whether your ears can hear all of it or not. Therefore, --insane "wastes" bits that could otherwise be applied to other parts of the audible frequency spectrum. That's why I modified --xtreme rather than --insane: optimization.

--xtreme produces smaller files than --insane, so generally --insane sounds "better" by default. However, the --nmt and --tmn switches allow you to increase headroom beyond a preset's default setting (thereby eating up a lot more bits). Indeed, --xtreme --nmt 16 --tmn 32 produces files that are larger than --insane. That's because --insane's default nmt and tmn settings are lower (i.e. less sensitive, requiring less bits).

Yes, my settings produce files that are measurably better than --insane. It uses more bits and allocates them more optimally. But that's not to say that --insane is inferior from an audible perspective. I'm not claiming that I can hear a difference between --insane and my settings...I use these settings because I don't mind the larger files and know that I have files that are well-suited for transcoding, should the future need arise.
Title: Ogg+Flac as lossy+recovery file!
Post by: tangent on 2002-02-14 17:45:21
lossy+recovery is not useless. There are definitely uses for this. Wavpack, for example has an implementation of this. The uses of this idea has been discussed before too. I admit it is not that useful compared to many other things, definitely not to the point of being crucial, but it is useful nevertheless.

@Benjamin
What you described is a perfect optimal implementation of bitrate peeling, but unfortunately it won't be that simple. You can't peel a -q5 vorbis stream down to -q3 bitrate and expect it to sound as good as a -q3 encode, but it won't be that far behind. The way I understand how bitrate peeling works is that the bits in the frames are packed so that the most significant information are packed at the start of the frame and the least significant information at the end of the frame. When you need to peel, you can chop off or truncate the end of the frame and lose only the least significant information, and the stream plays back without any problems.
Title: Ogg+Flac as lossy+recovery file!
Post by: Benjamin Lebsanft on 2002-02-14 18:08:43
i did not explain anything
Title: Ogg+Flac as lossy+recovery file!
Post by: Gecko on 2002-02-14 18:35:59
@mithrandir:

I agree that xtreme is more efficient than insane. But since insane stores the whole spectrum, Dibrom pointed out (it could be the same thread you were referring to) that this would in fact lend itself very well to transcoding while xtreme may do worse. On the other hand I do believe that at your extreme settings the difference is neglegible. You could of course also argue the other way around: using insane instead of xtreme with your tmn nmt values won't hurt either. Whatever suits you best.
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-14 19:14:48
Arrrrgh... this debate is going in circles.

Quote
Originally posted by kritip
What happens if you lose all your ogg's (recovery files) in the event of a hard disk faliure, won't the archived files then be of no use because they can't be recovered??


Yes, this has been pointed out repeatedly, and acknoledged by all parties involved. Please people, read the whole thread before posting.

Quote
Originally posted by mithrandir 
Sometimes we need to take a step back and ask "what exactly are we doing?"[/b]


So what you're criticizing isn't the lossy+correction file method of lossless backup, but the idea of any lossless backup at all. This, I think, is silly. With current hard drive sizes, large though they are, archiving a collection on your primary drive in a lossless format is overkill, I agree. I also agree that storing lossless backups on CDR is not too practical, only being able to store ~3 albums per CDR or so. If, however, I had a DVD-RW drive, or one of those 30gb tape drives, this idea would be very appealing.

Quote
Originally posted by mithrandir 
To go through this effort of making two files with two separate encoders, then burning one of them to media that can only handle 700MB...


I'll give you the point about burning to CDR, but as to making 2 separate files... a 2 line batch file or perl script would reduce this task to triviality.

Quote
Originally posted by mithrandir
If my original album were destroyed and I could never, ever get another original copy, I probably wouldn't "care" from an audio standpoint. What am I losing? Some wispy fine detail -60dB down in level that my ears can't hear anyway? Oh well, life goes on....


You're only focusing on the one criterion which is important to you: quality of the first generation lossy copy. For me there's more to it than that. Yes, MPC would give essentially transparent results... but how about MPC transcoded to MP3? *I* need to be able to play my files on a hardware player. As far as I know, no extensive testing has been done on the quality resulting from various transcoding scenarios, though it is guranteed to be less than a direct encode from original wav's. So OGG for me would probably be the best bet, as hardware player support has been promised by multiple vendors. OGG isn't quite as transparent yet as MPC though... so should I wait till 1.0 comes out and hopes it approches MPC levels of transparency? Even then, what if hardware support never materializes? So should I encode in MP3, knowing full well that even with Dibrom's improvements, my files will not be totally transparent?

Basically the problem isn't with these particulars, but with the fact that you're critcizing other's archiving methodology based on *your* requirements, rather than theirs. You need to take a long view and realize that the vast majority of people out there think that all of us who spend time on hyrdogenaudio and the like are crazy and wasting our time, since 128kbps is CD quality after all.
Title: Ogg+Flac as lossy+recovery file!
Post by: ssamadhi97 on 2002-02-14 19:58:34
lossless compression in general (and those lossy thumbs+recovery too) is very useful for trading in-concert recordings (too bad that shorten dominates there  )

but I think the attempt to use a common lossy format is dangerous since you usually can't rely on getting bit-identical output from different decoders/decoder versions...

edit: and you'd better watch out for limiting/replaygain/decoder-side clipping

(OT)
Quote
Tonight I encoded Live's A Distance To Here


way to go, mithrandir, your taste in music is good
(/OT)
Title: Ogg+Flac as lossy+recovery file!
Post by: mithrandir on 2002-02-14 20:18:34
Quote
Originally posted by Gecko
since insane stores the whole spectrum, Dibrom pointed out that this would in fact lend itself very well to transcoding while xtreme may do worse...using insane instead of xtreme with your tmn nmt values won't hurt either

I did investigate this and was a little surprised with what I found. I believe xtreme imposes a fixed lowpass of 19.5KHz but when I decode my xtreme files to WAVs and analyze them in a wave editor I notice that there is often information in the 20KHz+ region...sometimes even in the 21KHz+ region. When I use LAME aps, the frequencies get chopped off by 19KHz all the time, OTOH. Perhaps MPC's lowpass method is different than LAME's.

I did try my nmt/tmn settings with insane and bitrates generally went up by 15-20%. I decided it was not worth it considering that xtreme seems to have nearly full bandwidth anyway.
Quote
Originally posted by Randum
So what you're criticizing isn't the lossy+correction file method of lossless backup, but the idea of any lossless backup at all.

I actually trust the original pressed media more than burned dye CD-Rs, so I never seriously considered a backup plan. MAYBE for an album or two of mine that is rare, but in the worst case scenario, if I lose something I can generally rebuy it. Perhaps my logic is flawed, but I'm not losing any sleep over it.
Quote
Originally posted by Randum
Yes, MPC would give essentially transparent results... but how about MPC transcoded to MP3? *I* need to be able to play my files on a hardware player.

I suppose this is what I meant by "what exactly are we doing?"

If the purpose of transcoding my MPCs to MP3s is for playback on hardware players, then I don't see much of a problem at all. Why? Because these hardware players are devices used in environments that can hardly be considered conducive for audiophile playback. For example, if I must make a transcoded MP3 for use in a car MP3 receiver, what am I really losing? Unless you have a professional car stereo setup in a Lexus LS430, the noise from the road, the wind and the car itself will kill any of the little details that a super-high quality lossy encode may have preserved. Same thing with a portable like the Rio. If you listen on the go - on the train, on the bus, walking through town, whatever - are you going to be able to critically analyze the quality of the encode? Are you going to be able to tell the difference between a directly encoded MP3 and an MP3 transcoded from a high-bitrate MPC? On a portable or in the car, I should think not. Switching headphones may have a greater effect on sound quality. Is the portable device's output circuitry up to the task? Do you see my point? Diminishing returns. Obsessiveness over relative insignificance. I'm not trying to stunt progress but am suggesting that there may be other things of significance that deserve the effort.
Quote
Originally posted by Randum
you're critcizing other's archiving methodology based on *your* requirements, rather than theirs. You need to take a long view and realize that the vast majority of people out there think that all of us who spend time on hyrdogenaudio and the like are crazy and wasting our time, since 128kbps is CD quality after all.

It's not so much scathing criticism as it is an alternative perspective and a voice of reason. I well understand that most people here are very quality oriented. So am I. But I realize that you can get ahead of yourself in the search for perfection. I cannot say that what's "good enough" for me should be "good enough" for everyone else. But I do believe it is acceptable conversation to question methods and activities that others want to pursue...not necessarily to be condescending or elitist, but to present a perspective that adds to group thought processes and create a more well-rounded consensus.
Title: Ogg+Flac as lossy+recovery file!
Post by: SometimesWarrior on 2002-02-14 20:27:34
Even if this lossy + difference file method made the total file smaller (rather than larger) than just the lossless, I would not use it for backup purposes unless the lossy + difference file was significantly smaller.

Having a workable lossless file is much more useful; it can be played or re-encoded directly from the CD. Using the lossy + difference file requires temporary hard drive space to decompress and merge the files, not to mention a lot of extra processing and hard drive grinding time.

Also, I'd like to point out to Randum that Mithrandir did not criticize "the idea of any lossless backup at all." He says at the end of his post, "If [a] CD was known to be OOP/rare/etc, I would have used a lossless format like LPAC or MAC."

Personally, I think Mithrandir's backup methodology makes perfect sense for my needs, but then again there are people such as Randum and myself who want to be able to re-encode to other formats for portables or formats without the hassle of re-ripping.

Since my portable player is used in less-than-ideal playback situations, I have no problem with transcoding from "MPC --insane" to "Lame --ap fast standard" or "AACenc --normal." For those not willing to make the slight quality sacrifice, though, lossless is a viable option.

I've done my own tests, transcoding MPC > MP3 and MP3 > MP3, and I found that going from MPC --insane to Lame --ap 128 sounded like a direct --ap 128 encode, while Lame --ap standard to Lame --ap 128 had noticeable degradation. My ears aren't golden (both --insane and --ap standard sounded perfect to me), but as Dibrom and others have said, MPC is quite good for transcoding (to MP3, at least).

Edit: it looks like it took me a bit too long to type this: everything I had to say has already been said
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-14 20:37:02
Quote
Originally posted by mithrandir

I actually trust the original pressed media more than burned dye CD-Rs, so I never seriously considered a backup plan.

Pressed CD's do last longer than burned CD's.... if they are both stored in the same way. But audio CD's get tossed around, sat on, left under the seat of your car for weeks.... etc. Backups would sit in jewel cases in a box in my closet. Also, keep in mind that a large portion of music I would be 'backing up' I wouldn't have access to the pressed CD for longer than it takes to get it to the nearest computer with EAC

Quote
Originally posted by mithrandir

Because these hardware players are devices used in environments that can hardly be considered conducive for audiophile playback.


For most people, probably. For me, exactly the opposite. My car has $1200 of audio equiment in it, and I am in the process of dynamating the interior. I frequently hear details in music when listening in the car that I don't hear elsewhere. If I had a handheld, I would be using it with my $200 Sennheiser cans, the closest to optimal listening environment you can get on a non-ridiculous budget. My home speakers OTOH, are shitty labtec $60 computer speakers - it just isn't worth investing in good home speakers when I live in an apartment and can't turn em up loud. So for me, I need HIGHGER quality on a portable than on my desktop.
Title: Ogg+Flac as lossy+recovery file!
Post by: CiTay on 2002-02-14 20:38:54
Quote
Originally posted by mithrandir

I did investigate this and was a little surprised with what I found. I believe xtreme imposes a fixed lowpass of 19.5KHz but when I decode my xtreme files to WAVs and analyze them in a wave editor I notice that there is often information in the 20KHz+ region...sometimes even in the 21KHz+ region. When I use LAME aps, the frequencies get chopped off by 19KHz all the time, OTOH. Perhaps MPC's lowpass method is different than LAME's.


Yes, MPC uses a variable lowpass in --xtreme. The lowpassing threshold is adjusted on-the-fly during encoding, depending on the HF energy and psychoacoustic calculations.
Title: Ogg+Flac as lossy+recovery file!
Post by: Dibrom on 2002-02-14 21:41:41
Quote
Originally posted by CiTay
Yes, MPC uses a variable lowpass in --xtreme. The lowpassing threshold is adjusted on-the-fly during encoding, depending on the HF energy and psychoacoustic calculations.


This is sort of correct, except that it isn't actually lowpassing I think.  Instead it's just all based on the ath and various masking calculations, so in the end it has a similar effect to an adaptive lowpass.. just a nitpick I suppose, but an explicit lowpass would be a bit different than just not encoding up to a certain frequency cutoff which all beyond is deemed inaudible by the psymodel.
Title: Ogg+Flac as lossy+recovery file!
Post by: meff on 2002-02-15 17:42:21
isn't the bottom line is that:

this produces 2 files, one with the whole song encoded in ogg, and another with the .wav difference lossless encoded.

therefor, the total encode for backup purposes is smaller than just the full lossless encode, PLUS the added bonus of a easily distributable lossy version.

i don't know about you guys, but this seems like it has alot of opportunitys in various areas.

if your archiving, the end result is smaller.
and you have something you can toss to a friend!

pretty cool if you ask me.

would be the most useful with a DAT tape drive
Title: Ogg+Flac as lossy+recovery file!
Post by: JohnV on 2002-02-15 18:27:32
Quote
Originally posted by meff
isn't the bottom line is that:

this produces 2 files, one with the whole song encoded in ogg, and another with the .wav difference lossless encoded.

therefor, the total encode for backup purposes is smaller than just the full lossless encode, PLUS the added bonus of a easily distributable lossy version.

i don't know about you guys, but this seems like it has alot of opportunitys in various areas.

if your archiving, the end result is smaller.
No. This is just it. The result will not be smaller than just one lossless encode.
Ogg+lossless difference signal is always bigger than just single full lossless original.
Title: Ogg+Flac as lossy+recovery file!
Post by: Randum on 2002-02-15 18:47:44
Quote
Originally posted by JohnV
No. This is just it. The result will not be smaller than just one lossless encode.
Ogg+lossless difference signal is always bigger than just single full lossless original.

No, you guys are still not getting the point of all this. Yes, lossy+correction file will be bigger than lossless alone... but it will be smaller than lossy+lossless... which is what you would need if your situation requires a lossless backup, and easy access to a playable version.
Title: Ogg+Flac as lossy+recovery file!
Post by: JohnV on 2002-02-15 19:05:22
Yeah I get it. I just didn't pay enough attention what was said..
Somehow I totally missed the "PLUS the added bonus of a easily distributable lossy version." -part.
Title: Ogg+Flac as lossy+recovery file!
Post by: ssamadhi97 on 2002-02-16 01:20:32
I wonder whether there's a possibility to compress the difference file more efficiently...

lower amplitude, but higher enthropy, hmmm :confused: ideas, anyone?
Title: Ogg+Flac as lossy+recovery file!
Post by: lucpes on 2002-02-16 10:12:43
Quote
Originally posted by ssamadhi97
I wonder whether there's a possibility to compress the difference file more efficiently... 

lower amplitude, but higher enthropy, hmmm :confused: ideas, anyone?


My assumption is that "more efficiently" would mean lossy, and once you'll get here you'll end up with nothing else but garbage in the end...

Anyway, I see no point in storing the difference other than for "academic purposes" (What if the HDD crashes??)... recordable CD's are very cheap these days...
Title: Ogg+Flac as lossy+recovery file!
Post by: Ammethyl on 2002-02-21 17:48:19
Well, I've been experimenting with deltaS (the leftover) since I'm trying to find the most optimal way to store field recordings for my samples. I record a lot, but sometimes just don't have the time to manipulate. So I store the result, until I at least have the time to edit and take off the garbage (the long useless parts of a 2 hours recording....) In my case, I was particularly interested in compression closest possible to "bit perfect", since the material will, later, be manipulated and that psymodel can ruin an "artifact" that I may want to isolate later...

Someone have suggested that Lossless Audio compressor (Monkey's, flac, shorten and friends) MIGHT not be the most efficient for the nature of the data contained in a leftover file. He was suggesting that a general purpose compressor would PROBABLY be better than one specialised with AUDIO.  (In fact that was an interesting idea, because it's pretty true that DeltaS (leftovers) sound like fuzz or filtered white noise,; and the higher the bitrate of the lossy codec, the more fuzzy (and small) is the leftover. Try with Lame --freeformat 500kbps, fuzz-fuzz-fuzz.

I tried, yet only once, to compress the leftover with zip instead of Monkey . Here's the result:

Aphex Twin- I care because you do- 08 - Wet tip hen ax compressed with lame --alt-preset lowpass18 -F the result was 8 592 KB (original 54 655kb) the leftover compressed, in Monkeys audio at 43% the original size and in winzip, 55%, so I believe that Audio compressor are still more suited for leftovers than Genral purpose one. I may théorize that even if the result (leftover) sounds like fuzz, the samples in it still bear a strong correlation, that Monkey's audio is more able to exploit than WinZip.

I would permit myself to add this note: Compression is still FAR from being pushed at it's max, we will see lossless Audio Compressor that will perform at ratios like 40 or more, just because Music is a lot simpler than we may think, there is so much redundency that today's Codecs and compressors don't perceive.  Both Codec and compressor look at Audio data with a microscope, searching for extremely local redundency. They incorporate not much AI (if at all), they don't understand the nature of the sounds they compress....  Just look at MPEG4-SA  (structured audio) the plan is to determine what instruments are played, in what way, the recreate it in a midi-like way, and then, to correct the mistakes that have been made.... Pretty powerfull and extremely scalable....Better than CD quality at 32kbps....