Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: New OptimFROG (+ library) version released! (Read 31784 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

New OptimFROG (+ library) version released!

Hello,

I have great news for OptimFROG users: OptimFROG pr4.510,
and the Win32+Linux library pr1.200 are ready and can be
downloaded (on the Download page) from the OptimFROG site
(which is lately http://www.LosslessAudio.org).

Please let me know if you find any problems so I can make
this a release version

What is new in this version:
  - encoding is now 6% faster with exactly the same
    compression
  - all the source code was reorganized and rewritten
    to use uniform standards and increase portability
  - OFR, OFS, OFF and the DLL/SO library were merged
    into a single, common code base
  - significantly reduced system dependent code areas
  - all programs are now Valgrind clean
  - fixed small problems (like not setting the file
    date for the correction file)

  - the OptimFROG library is now available for Linux
    as a SO library
  - the OptimFROG.dll version 1.200 is binary
    compatible with the previous versions
  - added new function OptimFROG_freeTags to release
    the memory allocated for the tags structure
  - new C++ wrapper interface for the OptimFROG
    library requiring less work for writing plug-ins
  - two C usage examples and two C++ usage examples
    of the library for Windows/Linux

  - complete source code for dBpowerAMP, Winamp 5,
    foobar2000, and XMMS plug-ins

  - site address has changed to (the other are still
    usable) http://www.LosslessAudio.org

What to expect next:
  - development of OptimFROG will greatly accelerate
  - next major version will most probably include:
    improved compression, creation of self extracting
    archives, ID3v2 tags, Darwin/MacOS port, recovery
    information, gapless joining of multiple OptimFROG
    files, recovery of data from severely broken files

What is needed (what you can contribute):
  - it would be great to have extended support (more
    plug-ins for other players, sound editing tools)


If you have any questions, comments, suggestions or problems
regarding OptimFROG please don't hesitate to contact me at:

  FlorinGhido@yahoo.com

You can always find the newest version of OptimFROG at:

  http://www.LosslessAudio.org

P.S. I would liked to put this in validated news, but I didn't
have enough rights

Thank you,
Florin Ghido

New OptimFROG (+ library) version released!

Reply #1 –
Excellent and good to hear from you again!

New OptimFROG (+ library) version released!

Reply #2 –
Quote
OFR, OFS, OFF

a lil text about what each exe is supposed to do would be very nice (i just wanna use the lossless option for example),

thanks for the new release.
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

New OptimFROG (+ library) version released!

Reply #3 –
 Wonderfull, thank you and keep up the great work Florin!

@smok3: OFR = lossless encoder, OFS = DualStream encoder, OFF = IEEE Float Audio encoder, they're described in the Doc folder of the archive.

New OptimFROG (+ library) version released!

Reply #4 –
Would be great if http://www.rockbox.org/ supported this format

New OptimFROG (+ library) version released!

Reply #5 –
Quote
Would be great if http://www.rockbox.org/ supported this format [a href="index.php?act=findpost&pid=314722"][{POST_SNAPBACK}][/a]


No chance without sources.

And even with sources chances would be smaller than 0. Or anyone here expects OptimFrog to decode at real time with 140mHz?

New OptimFROG (+ library) version released!

Reply #6 –
Quote
And even with sources chances would be smaller than 0. Or anyone here expects OptimFrog to decode at real time with 140mHz?
[{POST_SNAPBACK}][/a]

According to [a href="http://web.inter.nl.net/users/hvdh/lossless/All.htm]Hans comparison[/url], in Fast mode with 900MHz it decodes at about 15x, so proportionally you need about 60MHz to decode at realtime. If you additionally take into account that with proper settings you can make decoding even faster, and that CPU in portable may be in decoding more efficient per MHz than AMD or Intel CPU, then the chances grow up to more than 0 [:

New OptimFROG (+ library) version released!

Reply #7 –
Quote
Hans comparison, in Fast mode with 900MHz it decodes at about 15x, so proportionally you need about 60MHz to decode at realtime. If you additionally take into account that with proper settings you can make decoding even faster, and that CPU in portable may be in decoding more efficient per MHz than AMD or Intel CPU, then the chances grow up to more than 0 [:[a href="index.php?act=findpost&pid=314734"][{POST_SNAPBACK}][/a]


Actually, in that case, AMD or Intel will be more efficient than the ColdFire. AFAIK, OptimFrog uses x86 ASM intensively to reach good speed on the x86 platform. The ColdFire CPU being used in the iRiver is not based on x86, but M68k instead.

Also, proportionality doesn't work that way when you compare different architectures.

For instance, look at FLAC in Heijden's comparison. It decodes 50x faster than real time. But it won't decode at 18mHz on the iRiver, no matter how much people would like it to happen.

Last but not least, this whole discussion is a moot point unless Ghido opens his sources. The RockBox is GPLd.

New OptimFROG (+ library) version released!

Reply #8 –
Quote
AFAIK, OptimFrog uses x86 ASM intensively to reach good speed on the x86 platform. The ColdFire CPU being used in the iRiver is not based on x86, but M68k instead.

M68k architecture is (or rather was) generally more efficient than x86 one. And to have it working on ColdFire you would need to port it anyway, so it would be a question of proper porting to have it more or at least as efficient.
Quote
this whole discussion is a moot point unless Ghido opens his sources.
[a href="index.php?act=findpost&pid=314739"][{POST_SNAPBACK}][/a]

True. Though from my personal POV (I'm not using portables) I prefer it to be closed-source than to watch zillions of potential forks (especially that Florin is doing a good job at maintaining and adding all the stuff).
 

New OptimFROG (+ library) version released!

Reply #9 –
Quote
True. Though from my personal POV (I'm not using portables) I prefer it to be closed-source than to watch zillions of potential forks.[a href="index.php?act=findpost&pid=314745"][{POST_SNAPBACK}][/a]


That is bullshit. FLAC, WavPack and Monkey's are open source, and you don't see forks.

Vorbis is open source, there are forks, and these forks are all for the best.

That argument reminds me of the old argument used by Musepack fanatics. "Since sources are not open, nobody can take them and f*** them up, creating bad quality files". What problem would be if people messed the sources, if we still have a trusted place to get good binaries?

Quote
M68k architecture is (or rather was) generally more efficient than x86 one.


As I pointed out already, you're basing your conclusions on a proportionality that doesn't necessarily exist. You can't safely extrapolate results in Heijden's test to other clock speeds, let alone other architectures!

New OptimFROG (+ library) version released!

Reply #10 –
Quote
Quote
True. Though from my personal POV (I'm not using portables) I prefer it to be closed-source than to watch zillions of potential forks.[a href="index.php?act=findpost&pid=314745"][{POST_SNAPBACK}][/a]

That is bullshit.

Oh come on, every sane person knows that open-source has both good & bad sides, for developers as well as for users. Foobar2000 isn't kept closed without a reason, even if Linux or MAC users would want to have it too.
Quote
As I pointed out already, you're basing your conclusions on a proportionality that doesn't necessarily exist. You can't safely extrapolate results in Heijden's test to other clock speeds, let alone other architectures!
[a href="index.php?act=findpost&pid=314747"][{POST_SNAPBACK}][/a]

You can't either. It was you who stated that it's impossible to decode OptimFROG at 140MHz in realtime. My statement that it could be decoded at even slower speeds is as right/wrong as yours.

Anyway, I find both things rather senseless to discuss here further.

New OptimFROG (+ library) version released!

Reply #11 –
Quote
I have great news for OptimFROG users: OptimFROG pr4.510,
and the Win32+Linux library pr1.200 are ready and can be
downloaded (on the Download page) from the OptimFROG site
(which is lately http://www.LosslessAudio.org).[a href="index.php?act=findpost&pid=314574"][{POST_SNAPBACK}][/a]
It's great to see one of the top lossless compressors actively being developed. With the continuous work being done on the portable aimed FLAC, WavPack, etc., I was afraid La, Monkey's and OptimFROG would start lagging. Glad to see that doesn't go for the latter at least.

Quote
What is new in this version:
  - encoding is now 6% faster with exactly the same compression[a href="index.php?act=findpost&pid=314574"][{POST_SNAPBACK}][/a]
Great work, because you know even better than anyone else that speed was and is OF's major disadvantage.

Quote
What to expect next:
  - development of OptimFROG will greatly accelerate
  - next major version will most probably include: improved compression, ...[a href="index.php?act=findpost&pid=314574"][{POST_SNAPBACK}][/a]
Really looking forward to the day when OptimFROG actually outperforms La on both ratio and speed.

Congratulations on the new version and keep those updates coming

New OptimFROG (+ library) version released!

Reply #12 –
Quote
Oh come on, every sane person knows that open-source has both good & bad sides, for developers as well as for users. Foobar2000 isn't kept closed without a reason, even if Linux or MAC users would want to have it too.


The fact is, what would be the point of forking OptimFrog? Creating a version that compresses better? In that case, all the better! Let it come.

Quote
Anyway, I find both things rather senseless to discuss here further.[a href="index.php?act=findpost&pid=314752"][{POST_SNAPBACK}][/a]


Tough.

New OptimFROG (+ library) version released!

Reply #13 –
Quote
M68k architecture is (or rather was) generally more efficient than x86 one.
[a href="index.php?act=findpost&pid=314745"][{POST_SNAPBACK}][/a]

Except for floating point 
In theory, there is no difference between theory and practice. In practice there is.

New OptimFROG (+ library) version released!

Reply #14 –
Quote
Really looking forward to the day when OptimFROG actually outperforms La on both ratio and speed.
[a href="index.php?act=findpost&pid=314754"][{POST_SNAPBACK}][/a]

Well, OptimFrog seriously outperforms LA on speed, and LA does not give that much better ratio. When you add the facts that LA are buggy, featureless and seemingly unmantained, OptimFrog won't be hard to choose for those situations when your looking for that maximum compression....

Tried the foobar and Winamp plugins. The foobar one seems to work well, but the Winamp one crashes both Winamp5 and 1by1. But as files that are compressed with the new encoder plays back well with the 1.07 Winamp2 plugin who works flawlessly with both players, it's not really a problem for me....

I'm on XPsp2. If you need specific info, please tell...
"ONLY THOSE WHO ATTEMPT THE IMPOSSIBLE WILL ACHIEVE THE ABSURD"
        - Oceania Association of Autonomous Astronauts

New OptimFROG (+ library) version released!

Reply #15 –
Quote
Tried the foobar and Winamp plugins. The foobar one seems to work well, but the Winamp one crashes both Winamp5 and 1by1. But as files that are compressed with the new encoder plays back well with the 1.07 Winamp2 plugin who works flawlessly with both players, it's not really a problem for me....
I'm on XPsp2. If you need specific info, please tell...

For Winamp 5, the OptimFROG.dll can be used from the following locations, in this order (the base paths can be different; I am running under Win2000):
(a) d:\Program Files\Winamp\
(b) d:\WINNT\system32\

The first is the directory where the winamp.exe is located, the other is a system-wide DLL search location. I tried playing .ofr and .ofs files after copying the DLL in (a) or (b) and it works ok.
I used the in_ofr.dll and OptimFROG.dll from OptimFROG_SDK_All_pr1200.zip.
When you try to use OptimFROG.dll version 1.100 (previous version), here on Win2000, the plug-in fails to run (it's not loaded at all) because the procedure entry point OptimFROG_freeTags is not present, but it does not crash though.

You can use OptimFROG_SDK\Test_SimpleC\OFROGDecSim.exe with the old OptimFROG.dll, version 1.100, from the OptimFROG_SDK_Win32_1100.zip, to see if it crashes; it should just display a startup error message.

You can also try deleting OptimFROG.dll from the (a) and (b) locations and copy again the new one, but this would not explain why it worked that way... Could you tell me a little more about how exactly Winamp crashes?

Thanks,
Florin Ghido

New OptimFROG (+ library) version released!

Reply #16 –
Quote
Well, OptimFrog seriously outperforms LA on speed, and LA does not give that much better ratio.[{POST_SNAPBACK}][/a]
Wonder what gives you that idea. Take a look at the much quoted performance data by [a href="http://web.inter.nl.net/users/hvdh/lossless/lossless.htm]HansHeijden[/url], Speek and Guruboolez.
For a given compresion ratio, OptimFrog is twice as slow as La, both at encoding and at decoding (and even slower compared to Monkey's Audio, for that matter). La is only equalled in ratio by OptimFrog's very highest (and slowest) extranew/bestnew setting, and then still, only in Guruboolez' tests.

New OptimFROG (+ library) version released!

Reply #17 –
Quote
Quote
Well, OptimFrog seriously outperforms LA on speed, and LA does not give that much better ratio.[{POST_SNAPBACK}][/a]
Wonder what gives you that idea. Take a look at the much quoted performance data by [a href="http://web.inter.nl.net/users/hvdh/lossless/lossless.htm]HansHeijden[/url], Speek and Guruboolez.
For a given compresion ratio, OptimFrog is twice as slow as La, both at encoding and at decoding (and even slower compared to Monkey's Audio, for that matter). La is only equalled in ratio by OptimFrog's very highest (and slowest) extranew/bestnew setting, and then still, only in Guruboolez' tests.
[a href="index.php?act=findpost&pid=315277"][{POST_SNAPBACK}][/a]


This is slightly misleading. All the above sites use for comparison Duron processors at 800/900 MHz, which don't have SSE. LA and Monkey's Audio use MMX for their computations, and without MMX they would be 3 times slower than now. On the other hand, OptimFROG uses SSE (which is present starting with Athlon XP and P3), when available, to speed up its computations. So, it is unfair to just say that OptimFROG it's twice slower than LA. None of the comparison sites assert the relations between OptimFROG's highnew, extranew, bestnew and LA's normal and high; they have only some combinations, which do not make clear the relation between compression ratios. Also, the test databases are somehow little, and not comprising all genres.

Another fact is that if you go to 24 bit, LA does not work at all, and MAC has low performance (its predictors are degenerated), due to them being tied to the MMX technology. OptimFROG fully supports it, at unchanged speeds (compared to 16 bit).

Here are, in my opinion, more accurate results, based on the corpus of 1047 files, having a total of 46992643860 bytes (46.993 GB), containing all genres, CD audio data, which I use for regressions and benchmarks.

52.70% OptimFROG bestnew 4.510

53.52% La high 0.4b
53.65% OptimFROG extranew 4.510

53.75% La normal 0.4b
53.78% OptimFROG highnew 4.510

As a conclusion: in my opinion, you can practically consider equivalent La high and OptimFROG extranew, and respectively, La normal and OptimFROG highnew. Also, OptimFROG bestnew is better here than La high by 0.82%.

About the time, here are some results on my Amd Athlon XP 1800+ for decoding speeds (the most important):

1.90x OptimFROG bestnew 4.510 [encoding 1.17x]

3.59x La high 0.4b [encoding 3.04x]
3.35x OptimFROG extranew 4.510 [encoding 2.09x]

5.06x La normal 0.4b [encoding 4.02x]
4.01x OptimFROG highnew 4.510 [encoding 2.67x]

My point is that OptimFROG will never fail badly. Every time, you can be sure that it is at least close the best value. I have samples recorded from the radio, on which LA (and Monkey's Audio, on which is based) are not able to track the complex stereo dependencies, and fail with 10-15% worse compression than OptimFROG.

OptimFROG is the result of many years of personal research, and is oriented towards the best compression possible. It has modes which work much slower than that of other compressors, but this do not imply it is a slow compressor. Its normal mode, is even a little faster at decoding than Monkey's Audio extra high mode, with comparable results (they are generally within 0.3%), only LA being significantly better.
So, it is not fair, in my opinion to say that OptimFROG is slow, only because it ALSO has modes which are slower.

As a note, I would like to mention that Monkey's Audio good performance (and implicitly LA) has at its roots the OptimFROG optimal stereo predictor concepts, somewhere in the beginning of 2002, when I explained them to Matthew. MAC uses a very simplified model of the optimal stereo predictor, but nevertheless, the compression increase was substantial (between versions 3.94 and 3.95).

Thank you,
Florin Ghido

New OptimFROG (+ library) version released!

Reply #18 –
That's most interesting to read, Florin. Thanks for that, and for OptimFROG.

In fact, as I mentioned before, it's a relief to finally see some movement at the upper end of the compression scale, now that La and Monkey's seem to be resting on their laurels.  Apple Lossless, FLAC or WavPack may be the codec of choice for playback and everyday use, but for long-time archival I prefer the sheer compression power of La or OptimFrog.

So keep it up and good luck!

New OptimFROG (+ library) version released!

Reply #19 –
hi,

i tested optimfrog and noticed something that i already saw with wavpack and which is different from flac for example.

i'm talking about the storage of the raw audio's checksum. after a quick test - corrupting an .ofr file - i realised that this checksum is not the main authority for error detection. i assume this is also the reason that it's not stored by default and has to be enabled by a command line switch? is there a reason to _not_ store it by default?

flac seems to treat the md5 checksum as an integral attribute of the encoded audio data, i.e. it's in the mandatory STREAMINFO tag and you cannot _not_ store it.

thanks

New OptimFROG (+ library) version released!

Reply #20 –
Quote
hi,

i tested optimfrog and noticed something that i already saw with wavpack and which is different from flac for example.

i'm talking about the storage of the raw audio's checksum. after a quick test - corrupting an .ofr file - i realised that this checksum is not the main authority for error detection. i assume this is also the reason that it's not stored by default and has to be enabled by a command line switch? is there a reason to _not_ store it by default?

flac seems to treat the md5 checksum as an integral attribute of the encoded audio data, i.e. it's in the mandatory STREAMINFO tag and you cannot _not_ store it.

thanks
[a href="index.php?act=findpost&pid=315543"][{POST_SNAPBACK}][/a]

I can't speak for Florin, but the reason that WavPack does not store the MD5 by default is simply to save the time of computing it (which gets to be significant, especially in the fast mode). WavPack (and OptimFROG) use a different error detection scheme internally on every block (which is required to silence errors during playback), so there's no need to do an MD5 sum by default to be robust.

In addition, with WavPack lossy (and maybe OptimFROG dualstream) the sum stored is computed on the original data, so it would not be useful for detecting errors at all in those modes.

I assume that FLAC has another error detection method for each block also, but since FLAC has no lossy mode it's not unreasonable for it to use a global MD5 for the whole file as an additional verification.

New OptimFROG (+ library) version released!

Reply #21 –
Quote
So, it is not fair, in my opinion to say that OptimFROG is slow, only because it ALSO has modes which are slower.


OptimFrog might not be slow per se (I'd concur it is), but when you compare with other formats like WavPack and FLAC, that encode several times faster, the comparision turns out to be obvious: OFR is a slow encoder. That's not a problem. It's like comparing ZIP and PAQar. ZIP is for acceptable compression and practicity. PAQar is for extreme compression at the cost of interoperability and, of course, time.

The slowness of OFR turns out to be more obvious when you compare it against Monkey's. Using Heijden's comparison (my favourite non-biased lossless comparison), you see that Monkey's in "normal" compress more and faster than OptimFrog's "fast". Same applies to APE's "extra high" versus OFR's "normal", "high" and maybe even "extra".

That's why my lossless comparison places OFR as "slow" next to LA.

New OptimFROG (+ library) version released!

Reply #22 –
Quote
I can't speak for Florin, but the reason that WavPack does not store the MD5 by default is simply to save the time of computing it (which gets to be significant, especially in the fast mode).


i can't quite follow. md5 computation should be dirt cheap. openssl speed md5 on the duron 1600 in my server box claims to do just over 270MB/s. my p4 2600 with hyperthreading does over 500MB/s. the numbers are for block sizes of 8K.

am i missing something?

edit:

ofr --encode: 19.54s user 0.94s system 97% cpu 21.082 total
ofr --encode --md5: 20.02s user 0.96s system 97% cpu 21.574 total

edit2:

about the lossy modes, that's obviously correct. i implied lossless-only. and yes, flac doesn't need the md5 to detect errors mid-stream also.

New OptimFROG (+ library) version released!

Reply #23 –
Quote
i can't quite follow. md5 computation should be dirt cheap. openssl speed md5 on the duron 1600 in my server box claims to do just over 270MB/s. my p4 2600 with hyperthreading does over 500MB/s. the numbers are for block sizes of 8K.

am i missing something?
[a href="index.php?act=findpost&pid=315556"][{POST_SNAPBACK}][/a]

Well, I guess you're missing that WavPack fast mode is dirt cheap also. 

I just measured 8% slower doing the MD5 in that mode, which is enough of a hit that I don't think everyone should pay. I guess I could have an option to turn off an option that most people don't use, but that seems backwards to me.

New OptimFROG (+ library) version released!

Reply #24 –
First, there is a new release of the SDK, updating only the binary libraries (DLL/SO/lib, no need to recompile anything) which fixes the problem presented by Mr_Rabid_Teddybear. He was kind to send me a DrWatson dump, which was very helpful, and I want to thank him for his support.
The problem was with deallocation of tags in some situations, causing a crash on Windows when OptimFROG_freeTags is called.

bryant: I'm glad to hear from you...

TCM: As bryant said, I consider that not everyone uses the MD5 value. As you just tested, there is a slight increase in time, by the use of MD5 computation (in your test, 21.574/21.082 -> 2.33% increase, greater for faster modes, smaller for slower modes).
OptimFROG has CRC32 checksums for every block of several seconds, and they are used to check for *compressed block corruption* starting with the writing to disk (which can be faulty in very rare situations). The MD5 can be used to fingerprint the audio data without decompressing it, and for ensuring it decodes correctly.
For checking against compressed block corruption, you can use --verify which is light fast. For checking against correct decoding, you can use --check which decodes the file in memory, does the MD5 and then, if there is a MD5 stored, it compares with it.

rjamorim: I agree with you that compared with WavPack and FLAC, OptimFROG is slower, as they are targeted towards different goals.
Indeed, OptimFROG normal compresses slower than MAC extra high, but decodes a little faster. In my tests, and also in my opinion, I think of those modes as being somehow equivalent, being generally within 0.3% for mixed genres.
To compare them, OFR normal is better at noisier genres, and MAC extra high is better at tonal genres. The explanation (from inside) is that OFR normal uses a small predictor order with optimal adaptation, and MAC uses a very large predictor order with suboptimal adaptation (as it uses the well known Sign Sign LMS algorithm with some scaling in the regressor).
About your lossless comparison pages, they are indeed an awesome resource...  [BTW, I agree perfectly with "quite slow" put there for OFR ]

Thank you,
Florin Ghido