lossless mp3

Topic: lossless mp3 - mp3HD (Read 106068 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

lossless mp3 - mp3HD

2009-03-19 16:33:47

Quote

mp3HD is a lossless audio codec (100% bit-exact replica of CD tracks)
Backward Compatible to mp3
File extension .mp3
Bitrates for music approximately 500 to 900 kbps rates (similar to other lossless codecs), depending on genre
Embedded mp3 track and the mp3HD file share the same id3 metadata
Encoding parameters (e.g. bit rate), ancillary data and meta data of embedded mp3 track are under control

Main page:
http://www.all4mp3.com/Learn_mp3_hd_1.aspx

...which links to downloadable test software:
http://www.all4mp3.com/Software3.aspx

I learnt about this from this post...
http://forum.doom9.org/showthread.php?p=1263465#post1263465
...where a quick test suggests it's not as efficient as FLAC, or as fast.

Might be useful for a widely-compatible lossy/lossless hybrid.

I wonder if they've done something clever - when I tried this years ago, mp3 was a terrible base for lossless coding - the lossy>lossless "correction" file was often larger than a straight lossless encode. Maybe this new technology is more tightly integrated into mp3, so performs better?

Cheers,
David.

lossless mp3 - mp3HD

Reply #1 – 2009-03-19 20:17:30

Quote from: 2Bdecided on 2009-03-19 16:33:47

I wonder if they've done something clever - when I tried this years ago, mp3 was a terrible base for lossless coding - the lossy>lossless "correction" file was often larger than a straight lossless encode. Maybe this new technology is more tightly integrated into mp3, so performs better?

Cheers,
David.

If I had to guess, I'd say that the mp3 encoder in question is designed to minimize the mathematical differences with the original file, as opposed to various modern mp3 codecs that are designed to minimize the acoustic differences.

And I question the backward compatibility. I encoded a 3 minute 11 second song and mplayer is reporting that it's 46:28 long.
~~And the quality of the "VBR highest quality" mode isn't particularly good. (ABX test to follow, I hope)~~
The efficiency isn't that good, either. That 3:11 song is approximately 1.3 MB bigger, too.

lossless mp3 - mp3HD

Reply #2 – 2009-03-19 20:27:07

This format must for the love of everything good not succeed. Please let it die. Do not even talk about it.
There is not reason to have a hybrid lossless/lossy format. And if you want that you already have WavPack's Hybrid Lossy mode.

lossless mp3 - mp3HD

Reply #3 – 2009-03-19 22:23:44

If it were really backward compatible, why does it need a plugin for playback? And why is it restricted to Windows. I'm gonna have to call BS on this one ...

lossless mp3 - mp3HD

Reply #4 – 2009-03-19 23:02:04

Quote from: LANjackal on 2009-03-19 22:23:44

If it were really backward compatible, why does it need a plugin for playback? And why is it restricted to Windows. I'm gonna have to call BS on this one ...

A specific decoder is certainly needed to decode the lossless stream ; but any standard decoder would be able to decode the lossy part - hence the need for a specific decoder to get the best quality.
It sounds interesting - at least if performance (ratio) is on par with other formats. It should at least seduce people dealing with two different libraries (lossless for home - lossy for car or DAP).

Was someone able to make it work with foobar2000's converter? I could only get a regular mp3 file (128 kbps according to filesize) /-(

lossless mp3 - mp3HD

Reply #5 – 2009-03-19 23:31:50

Quote from: guruboolez on 2009-03-19 23:02:04

Was someone able to make it work with foobar2000's converter? I could only get a regular mp3 file (128 kbps according to filesize) /-(

Strangely enough, it works with foobar2000's Converter if you force it to use another file extension. I told fb2k to give it an mp5 extension and the result was a fully functional MP3HD file which, after being renamed to *.mp3, could be played in fb2k (the normal MP3 part, that is) and decoded to the original Wave file by the MP3HD decoder.

lossless mp3 - mp3HD

Reply #6 – 2009-03-19 23:56:34

@Silversight
Good point! It works. I guess that tagging process is the cause of the problem. Once the "mp5" file is created and renamed to mp3 (at this moment, the file has no tags), and after you add some tags, the file is shrinked by foobar2000 and all the lossless part is gone (but the mp3 stream is still valid).
I'm not technician, but I guess that MP3HD file are built like: MP3 then CORRECTION or MP3 then LOSSLESS.

lossless mp3 - mp3HD

Reply #7 – 2009-03-20 01:36:21

Quote from: ameyer17 on 2009-03-19 20:17:30

If I had to guess, I'd say that the mp3 encoder in question is designed to minimize the mathematical differences with the original file, as opposed to various modern mp3 codecs that are designed to minimize the acoustic differences.

For what it's worth, there appears to be a 17 kHz lowpass, so perhaps not.

Quote from: guruboolez on 2009-03-19 23:02:04

It sounds interesting - at least if performance (ratio) is on par with other formats. It should at least seduce people dealing with two different libraries (lossless for home - lossy for car or DAP).

Yeah, the best of both worlds, bigger than flac and lossy (if not dealing with the mp3hd decoder)
The one song I encoded was approximately 800 kbps, whereas the flac -8 file is approximately 740 kbps.
Encoding speed seems to be about half that of flac.
Decoding speed is about 28x, whereas flac decodes at about 192x or so.

(All comparisons with flac use the 32 bit Linux version of mp3hd as downloaded from all4mp3.com and flac as packaged in the 64 bit version of Ubuntu 8.10 on a computer with an Intel E7400 processor and 4 GB of RAM)

lossless mp3 - mp3HD

Reply #8 – 2009-03-20 01:49:25

Awhile back I wondered who would be the first to implement MP3 with correction, and now it has happened.

It works as expected and all the media players have no problem playing the resulting MP3HD files. Original files are identical to MP3HD decodes. The MP3HD compresses around the level of FLAC -3, although much slower (as expected).

Interestingly, in my limited tests, the resulting filesize seems to go up with the lossy bitrate.

lossless mp3 - mp3HD

Reply #9 – 2009-03-20 12:01:54

I investigate a bit further with this encoder.
• First, tag edition is currently ruining the lossless (or correction) datas. That's why foobar2000 doesn't work without forcing another extension (like mp5): it adds tags at the end of the conversion and therefore deletes the all "HD" informations. I created several MP3HD files, and changed tags with fb2k, dbpoweramp and mp3tag: each time files were shrinked by ~80%
With previous versions of foobar2000, user can disable tag writing after conversion: the encoder is therefore functional even with "mp3" extension.

• Then, I tried to see if MP3HD append an independant lossless file to the MP3 stream or if it really corresponds to a correction stream (like with WavPack and OptimFrog). For that purpose, I encoded the same file with different lossy targets (128 kbps, 320 kbps and also the highest VBR mode). Measuring the size of the "HD" data is easy once you can easily get the "lossy" part (with a simple tag edition as mentioned before). The size of the "HD" data changes according to bitrate. As a conclusion, it seems logical that the HD part is really a correction part, not an independant lossless stream append the a common MP3 file. The higher the lossy bitrate is, the lower the correction part is. In other words, MP3HD would bring some efficiency over a dual library (MP3 + FLAC for example).

My tests were based on two radically different files. The first one is a big hard rock track (several tracks merged) with an important lossless bitrate (>1100 kbps); the second is a long stereo John Cage composition including a lot of silence part and therefore encoded at an amazing 194 kbps bitrate with flac.

Click to enlarge the table:

or direct link : http://img5.imageshack.us/img5/9286/mp3hd.png

As a conclusion, I would say that MP3HD brings interesting but not amazing performance for people dealing with an independant MP3library. But to be really interesting it needs a simple software tool able to transfer to any portable player the lossy part of the lossless file. Unfortunately, I wouldn't recommand anyone using this encoder at the moment as your lossless collection could too easily be lost with a basic masstagging operation.

lossless mp3 - mp3HD

Reply #10 – 2009-03-20 14:00:24

If you want to know how it works, look at MPEG-4 SLS, this is the same, but with MP3 as core codec. Indeed it will only give extra efficiency if you plan to have a library of both lossy and lossless versions.

lossless mp3 - mp3HD

Reply #11 – 2009-03-20 14:54:59

Thanks menno.
Before someone asks, MP3HD doesn't benefit from lossywav (resulting file was even bigger on my test).

I also noticed two different issues:
- conversion failed with long tracks (> 1 hour). I used fb2k as frontend.
- with long tracks and therefore huge amount of correction file, it could take some time before some program could access to tags (noticed it with foobar2000 and DbPowerAmp tag editor). I read in a french site that correction data are written as IDTag. Is it possible (ID3 size limitation maybe)?

lossless mp3 - mp3HD

Reply #12 – 2009-03-20 15:11:59

Quote from: guruboolez on 2009-03-20 14:54:59

Thanks menno.
Before someone asks, MP3HD doesn't benefit from lossywav (resulting file was even bigger on my test).

I also noticed two different issues:
- conversion failed with long tracks (> 1 hour). I used fb2k as frontend.
- with long tracks and therefore huge amount of correction file, it could take some time before some program could access to tags (noticed it with foobar2000 and DbPowerAmp tag editor). I read in a french site that correction data are written as IDTag. Is it possible (ID3 size limitation maybe)?

Yep, the correction data is in the ID3 tag, because mp3 doesn't allow framesizes big enough for this. Maybe freeformat would work, but I guess hardly any players would play the mp3 file that way. And it would be very difficult to extract the original MP3 data. In MPEG-4 SLS, the lossless data is simply stored in a separate track in the MP4 file.

lossless mp3 - mp3HD

Reply #13 – 2009-03-20 15:50:10

Thank you again menno for this clarification
I wonder how difficult it will be to make MP3 tagging programs fully compatible with MP3HD. And how many people will lose the correction data by using a wrong tagging program (including audio players adding stats or stuff like that).

In my opinion, MP3HD was really useful and maybe designed for popular stores (like amazon) to give them the opportunity to sell lossless music accessible to everyone (as many customers don't even know file extension, so program installation or file conversion is not an option for biggest stores). But the bad interaction with the current software environment (tagging program and into some extend media players) is IMO ruining a bit this idea.

lossless mp3 - mp3HD

Reply #14 – 2009-03-20 16:21:45

When many customers doesn't even know file extension why would they care if they have a lossless mp3hd? they just see .mp3 and wonder why it's so big. and that lossless part is just accesible with appropriate decoder, which has to be installed.
and I don't understand why it's mp3hd that gives stores like amazon that opportunity? they could also just use flac...

for me, reading this thread tells me that mp3hd is just useless. mp3 an already old and inferior compression format becoming lossless... oh well >_>

lossless mp3 - mp3HD

Reply #15 – 2009-03-20 16:34:25

Useless format:

The lossless part is stored in ID3v2 tags.
Size of ID3v2 tags is limited to 256MB by specifications; as a result, lossless part of an mp3hd file can't be larger than 256MB.

I wonder what those people will think of next. Maybe resurrect VQF or something.

Addendum:

Current tagging software isn't prepared to deal with this kind of situation, so you're going to see various disturbing behaviors such as:

Very slow tag updates (near-full-file-rewrite with each edit).
Heavy memory usage of tag editors.
Retagging stripping correction data.
Tag editing or even reading failures when approaching the 256MB limit because software will try to put each ID3v2 frame in a single memory block and allocating a single block of such size is likely to fail in 32-bit address space because of fragmentation issues.

lossless mp3 - mp3HD

Reply #16 – 2009-03-20 16:47:43

Quote from: Peter on 2009-03-20 16:34:25

Useless format

Totally agree. Not having separate correction files is pretty bad, anyway. With mp3hd you can fit just one album on a CDR for your car stereo in the worst case. That's not the point of mp3.

I would propose a generic correction file creator for all lossy formats ("DeltaFlac"? ). A very simple piece of software or plugin could decode a lossy stream (including gap correction) and generate a delta stream against the original PCM, which would then be FLAC compressed and stored somewhere safe (for the reconversion of a whole collection in 2025 to Super-AAC2-Pro-3000).

A realtime playback filter, e. g., for Foobar could just reverse the chain. Delta and lossy file could be matched by a simple tag for the case that filenames get messed up.

lossless mp3 - mp3HD

Reply #17 – 2009-03-20 16:59:21

Quote from: callisto on 2009-03-20 16:21:45

When many customers doesn't even know file extension why would they care if they have a lossless mp3hd?

Having few knowledge about computer things doesn't mean that people don't care about quality and would buy anything. If advertising on "CD-Quality" is now pointless and confusing (because stores were claiming it since the beginning) selling "HD MP3" and advertising on files "four time bigger" (hence better) makes things easy to understand even to the less-technician people on earth.

Quote

and that lossless part is just accesible with appropriate decoder, which has to be installed.

It doesn't matter people could really access to the correction stuff or not. Most people (even here) wouldn't notice the difference between a truncated "HD" stream at 256 kbps and the full lossless music. I'm pretty sure placebo effect will ensure people that their HD file are now much better than regular MP3, even if the stream is the same.

Quote

and I don't understand why it's mp3hd that gives stores like amazon that opportunity? they could also just use flac...

Yes, and having their hotline full of people unable to make their flac/alac/WMAlossless work on their computer, digital players or even both. Selling a format customers don't understand means losing customers (I saw it myself with some friends of mine trying to deal with WMA and AAC). MP3HD is at the moment the only lossless solution that give a sound (even a nice one at 256 kbps) working on all DAP players, DVD players and software players in the world. Without installing anything. That would make a big difference for several people on earth.
What's the most attractive in your opinion:
"Buy HD MUSIC. Cristal clear sound. Compatible with every player"
or
"Buy lossless music. CD Quality. Download the MP3 codec, the GUI and follow our two page tutorial to make it work"

Note that I'm not personaly fond of this format (crappy IDTag solution) but I'm just trying to show that the concept isn't useless.

lossless mp3 - mp3HD

Reply #18 – 2009-03-20 17:03:58

Quote from: Peter on 2009-03-20 16:34:25

Size of ID3v2 tags is limited to 256MB by specifications; as a result, lossless part of an mp3hd file can't be larger than 256MB.

I suspected it when the encoder returned an error on long files (55 minutes of low bitrate classical music was working but not 40 minutes of high bitrate rock music).

lossless mp3 - mp3HD

Reply #19 – 2009-03-20 17:04:38

Quote from: rpp3po on 2009-03-20 16:47:43

Quote from: Peter on 2009-03-20 16:34:25
Useless format

Totally agree. Not having separate correction files is pretty bad, anyway. With mp3hd you can fit just one album on a CDR for your car stereo in the worst case. That's not the point of mp3.

While I agree on the uselessness of putting the data in ID3 tags, I disagree about the file size argument. You can remove the correction data by removing 1 frame from the ID3v2 tag, this is much faster than re-encoding your lossless files, and you will just end up with a normal "small" mp3 file. The idea is to have a lossless + lossy file at a smaller datarate than a separate lossless and lossy file, without the need to have to encode your lossless local files for portable usage.

Quote

I would propose a generic correction file creator for all lossy formats ("DeltaFlac"? ). A very simple piece of software or plugin could decode a lossy stream (including gap correction) and generate a delta stream against the original PCM, which would then be FLAC compressed and stored somewhere safe (for the reconversion of a whole collection in 2025 to Super-AAC2-Pro-3000).

A realtime playback filter, e. g., for Foobar could just reverse the chain. Delta and lossy file could be matched by a simple tag for the case that filenames get messed up.

Lossy formats are hardly ever decoded deterministically, so your idea will never work (at least not losslessly). In SLS (so also mp3HD I assume) the original AAC/MP3 stream is not decoded by a normal lossy decoder.

lossless mp3 - mp3HD

Reply #20 – 2009-03-20 17:10:37

@guruboolez
Ok, I get your point...
But still think it's crap. And to have a huge file on your DAP where you just can play the lossy part is just silly^^

I stick with buying CDs that I can touch, browse the booklet and look at in my cd-shelf

lossless mp3 - mp3HD

Reply #21 – 2009-03-20 17:14:33

Quote from: callisto on 2009-03-20 17:10:37

@guruboolez
Ok, I get your point...
But still think it's crap. And to have a huge file on your DAP where you just can play the lossy part is just silly^^

You're supposed to remove the lossless correction part before copying the file to your DAP. Of course this requires knowledge from the user about the format and I think that's where it will go wrong in general.

lossless mp3 - mp3HD

Reply #22 – 2009-03-20 17:20:10

Quote from: callisto on 2009-03-20 17:10:37

And to have a huge file on your DAP where you just can play the lossy part is just silly^^

I understand your point but I don't think it's silly. Nowadays cheapest flash-memory based MP3 players are rather big (2 GB at least). People could store 4 albums (sometimes more), or 60 tracks. It's a honest performance. I also know people unable to fill their players with lossy stuff and even other who consider as silly to put so many music on a portable player.
I tend myself to maximize the capacity of my own players (including a 160GB iPod filled with 130 kbps music) but I learnt that many people acting differently.

lossless mp3 - mp3HD

Reply #23 – 2009-03-20 18:18:04

Quote from: rpp3po on 2009-03-20 16:47:43

I would propose a generic correction file creator for all lossy formats ("DeltaFlac"? ). A very simple piece of software or plugin could decode a lossy stream (including gap correction) and generate a delta stream against the original PCM, which would then be FLAC compressed and stored somewhere safe (for the reconversion of a whole collection in 2025 to Super-AAC2-Pro-3000).

Hmm, interesting. Have a regular <filename>.mp3, and an accompanying <filename>.dflac file. Old players, just use the .mp3, new players the .dflac, and takes it's metadata from the /mp3 and uses it as a base to perform the corrections on, provided in the .dflac.

Could be quite cool. And would be _very_ convenient for me.

lossless mp3 - mp3HD

Reply #24 – 2009-03-20 18:34:31

some test files encoded with flac -8: 477 MB, or 976kbps. Musepack --extreme: 102 MB (210 kbps); difference between lossless and lossy encoded with flac -8: 400 MB, or 819 kbps.
So, we can make MusepackHD!

Notice