While playing with MPEG ALS I also studyed currently existing compressors a bit, to see if they had optimizations or tricks that could be of interest for ALS.
In the FLAC code, I found some things that looked like bugs/oversights. I did some attempts to fix that, and the result appears to be positive.
This Windows binary should compress tighter than the reference encoder at all settings almost all of the time:
http://sjeng.org/ftp/lossless/flac.exe (http://sjeng.org/ftp/lossless/flac.exe)
Interesting. Do the new changes require more CPU power when decoding? I mean, are these changes going to have impact on iAudio players being able to play level 2 FLAC only?
The worst case is identical to the reference encoder.
The average case may be different - I did not investigate that.
Given that likely the player stutters when encountering the worst case, I think it should work fine. The files are fully FLAC-standard-compliant.
Looks good here. I did a quick comparison with -8:
File: Size 1.1.2: Size 1.1.2.1: % Decrease:
04-Aly_And_AJ-Something_More.flac 29357707 29270378 0.297
04-Fanny_Hamlin_and_Adam_Appel-Ready_For_Love.flac 21187038 21080676 0.502
04-McFly-That_Girl.flac 25878788 25767410 0.430
04-NSync-Merry_Christmas_Happy_Holidays.flac 32022324 31892606 0.405
04-Opeth-The_Drapery_Falls.flac 85449229 84950401 0.584
06-BBMak-I_Can_Tell.flac 30068568 29973771 0.315
06-McFly-Too_Close_For_Comfort.flac 36072742 35855022 0.604
08-Dream_Theater-Home.flac 97724239 97071027 0.668
09-Play-Another_Love_Story.flac 25598417 25544604 0.210
10-Utopia_Banished-Forshadowing_The_Endless_Quest.flac 57122311 56928830 0.339
(Total) 440481363 438334725 0.487
And the CRC of the decompressed files are all correct of course.
EDIT: Improved table alignment.
In the FLAC code, I found some things that looked like bugs/oversights. I did some attempts to fix that, and the result appears to be positive.[a href="index.php?act=findpost&pid=355783"][{POST_SNAPBACK}][/a]
Thanks Garf, but what exactly changed? We lready know the decoding speed might have been affected, but was the compression ratio improved at all? Could you be more specific?
Edit (Garf): Uh, sorry, I'm tired and I accidentally edited your post instead of making a new one.
I tested it on my old machine (Intel P4 1.5GHz "Williamette") and -7 and -8 doesn't work here. The encoder freezes at 98%.
But here is my comparison with the "original" flac 1.1.2:
(tested with my own testsample provided here: http://www.megaupload.com/de/?d=O6WOVNJ8 (http://www.megaupload.com/de/?d=O6WOVNJ8)
replaygained with foobar2000 to a today's usual dynamic of 96dB)
Testsystem:
CPU: 1.5GHz P4 "Williamette"
RAM: 512 MB SD-RAM PC133
HDD: WD 80GB 7.200upm
OS: WinXP SP2
Programm: foobar2000 0.8.3 (decoding: speed-meter)
=================================================================================
MODE SPEED x
ENCODE DECODE RATIO
FLAC 1.1.2
-0 25,95 40,43 68,06%
-1 25,43 39,60 66,55%
-2 23,13 40,00 65,89%
-3 21,94 36,56 66,43%
-4 19,90 35,88 64,80%
-5 17,94 36,58 64,30%
-6 17,60 36,23 64,21%
-7 7,82 36,92 64,09%
-8 6,20 35,88 64,05%
FLAC 1.1.2.1 optimized by Garf
-0 26,85 41,29 68,06%
-1 26,47 40,00 66,55%
-2 22,72 40,84 65,90%
-3 22.32 34,90 66,27%
-4 20,00 35,90 64,71%
-5 17,29 36,92 64,17%
-6 15,17 37,26 64,07%
-7 FREEZES PC AT 98%----
-8 FREEZES PC AT 98%----
Great, seems to be Yet Another Intel C Bug
Try with http://sjeng.org/ftp/lossless/flac_msvc.exe (http://sjeng.org/ftp/lossless/flac_msvc.exe)
Are you going to make the code or at least an overview of the changes available in case Josh wants to use your changes?
@Garf
Yes, flac_msvc is working, but it encodes a little bit slower than the other.
EDIT: (new test with the new flac_msvc)
Testsystem:
CPU: 1.5GHz P4 "Williamette"
RAM: 512 MB SD-RAM PC133
HDD: WD 80GB 7.200upm
OS: WinXP SP2
Programm: foobar2000 0.8.3 (decoded: speed-meter)
=================================================================================
MODE SPEED x
ENCODE DECODE RATIO
FLAC 1.1.2
-0 25,95 40,43 68,06%
-1 25,43 39,60 66,55%
-2 23,13 40,00 65,89%
-3 21,94 36,56 66,43%
-4 19,90 35,88 64,80%
-5 17,94 36,58 64,30%
-6 17,60 36,23 64,21%
-7 7,82 36,92 64,09%
-8 6,20 35,88 64,05%
FLAC_MSVC 1.1.2.1 optimized by Garf
-0 24,93 41,29 68,06%
-1 24,45 40,00 66,55%
-2 19,90 40,84 65,90%
-3 21,33 37,29 66,27%
-4 18,28 35,88 64,71%
-5 16,33 36,92 64,17%
-6 12,92 36,92 64,07%
-7 3,75 36,92 63,91%
-8 2,91 36,23 63,87%
Seems to be a slight improvement in compression ratio from -3 up to -8, without any real effect on decoding speed.
Great! So you submitted this to Josh, right? *me hopes for a OS X build*
I'd be interested to see it.
http://sjeng.org/ftp/lossless/flac.exe (http://sjeng.org/ftp/lossless/flac.exe)
what's this? version is 1.1.2.1... different from flac_msvc.exe?
http://sjeng.org/ftp/lossless/flac.exe (http://sjeng.org/ftp/lossless/flac.exe)
what's this? version is 1.1.2.1... different from flac_msvc.exe?
[a href="index.php?act=findpost&pid=356120"][{POST_SNAPBACK}][/a]
That's the original one he posted compiled with Intel compiler, later replaced with a MSVC compile due to the bug tedgo found.
http://sjeng.org/ftp/lossless/flac.exe (http://sjeng.org/ftp/lossless/flac.exe)
what's this? version is 1.1.2.1... different from flac_msvc.exe?
[a href="index.php?act=findpost&pid=356120"][{POST_SNAPBACK}][/a]
That's the original one he posted compiled with Intel compiler, later replaced with a MSVC compile due to the bug tedgo found.
[a href="index.php?act=findpost&pid=356132"][{POST_SNAPBACK}][/a]
thx. i understand
Updated. Again better compression, though improvement is quite small :-P
I also replaced the binaries, now just one MSVC binary that is hopefully bugfree.
There seems to be something wrong with this new flac.exe, because I get this:
D:\Temp>flac.exe
The system cannot execute the specified program.
WinXp SP2
It works fine on my system (used it in foobar2000).
Compression ratio is between 0.02 and 0.04% worse than the flac_msvc-build... But with a slight speed improvement.
EDIT:
Have i downloaded the wrong binary? Its stated as "20060107" in the files properties.
Wanna make an optimized version of the wavpack encoder? That would be really great.
Btw, where's the source?
There seems to be something wrong with this new flac.exe, because I get this:
D:\Temp>flac.exe
The system cannot execute the specified program.
WinXp SP2[a href="index.php?act=findpost&pid=356625"][{POST_SNAPBACK}][/a]
Very same problem here...
ASUS P5P800, Intel Pentium 4 Prescott 640, 2xDDR-SDRAM PC3200 - 512 MBytes, Windows XP Professional SP2 fully updated.
Sergio
well, it works here. do you have a msvcr80.dll C runtime library?
http://www.dependencywalker.com/ (http://www.dependencywalker.com/)
well, it works here. do you have a mcvcr80.dll C runtime library?
http://www.dependencywalker.com/ (http://www.dependencywalker.com/)
[a href="index.php?act=findpost&pid=356654"][{POST_SNAPBACK}][/a]
Bingo! It is missing!
What is it? Where can I get it? Can't it be statically loaded?
Many thanks!
Sergio
Edit: BTW, very nice tool, the one you pointed to!
The flac.exe works fine on my system and i don't have a mcvcr80.dll.
Or did you meant a ms[/u]vcr80.dll?
The flac.exe works fine on my system and i don't have a mcvcr80.dll.
Or did you meant a ms[/u]vcr80.dll?
[a href="index.php?act=findpost&pid=356657"][{POST_SNAPBACK}][/a]
Oh, yes! It is m
svcr80.dll!
Compression ratio is between 0.02 and 0.04% worse than the flac_msvc-build... But with a slight speed improvement.
The new one is worse for me also when comparing to the original build. Out of the 10 files I tested, only 2 were better with the newer one, the other 8 were worse. The overall size of all 10 files came out worse with the new one.
Ugh! I tested it on several files and got an improvement each time. Seems I didn't test enough. And I set the compiler wrong. I suck!
I will play a bit more with this and see what I can do.
Wanna make an optimized version of the wavpack encoder? That would be really great.
[a href="index.php?act=findpost&pid=356633"][{POST_SNAPBACK}][/a]
WavPack seems to be based on entrely different technology as FLAC. As far as I can see WavPack is using backwards predictors (like Monkey's Audio). FLAC is using forwards predictors and simple LPC analysis.
Besides, the WavPack code is already very optimized. Hard to improve there (I tried in the past).
Will youre improvements be merged with Flac or will it just be a fork?
Will youre improvements be merged with Flac or will it just be a fork?
[a href="index.php?act=findpost&pid=356756"][{POST_SNAPBACK}][/a]
Or neither. (It seems Garf keeps it as a closely guarded secret instead of discussing it with fellas who know FLAC.)
Triza
Hellooooo, can we have the source code please? FLAC is distributed under the GPL license, if I'm not mistaking, and you're supposed to distribute the source code of the changes you made along with the binaries.
http://flac.sourceforge.net/license.html (http://flac.sourceforge.net/license.html)
The encoder license is basically the same as BSD I think so you don't need to release the source. Thats one of the reasons it was accepted for so many hardware projects. Would be nice to see it anyway though
Think im correct anyway
Kristian
The encoder license is basically the same as BSD I think so you don't need to release the source.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=357600")
That would be true for the libraries.
The rest of the software that the FLAC project provides is licensed under the GNU General Public License (GPL).
I can only assume that the [a href="http://sjeng.org/ftp/lossless/flac.exe]binary[/url] that Garf posted contains parts, if not most of, the GPL'ed command line application.
Anyway, IANAL, and I'm not interested in trolling about Free Software, but right now he only posted Windows binaries, while some people (such as myself) are running something else (linux in my case). It just sucks that Garf didn't provide the source code of his modifications, whether he is obliged to do it or not.
If you run flac (the original reference binary) without any options, it mentions the GPL. The main source file for flac mentions the GPL too.
Garf, could you upload your sources (or patches) to the same location, please?
FLAC libraries are BSD licensed.
FLAC frontend is GPL. The frontend has no changes[1]. Does linking the GPL frontend with the BSD libraries force you to publish the source for the libraries, or just for the frontend?
My impression was that it was needed for the frontend only. Since it's unchanged, you can download the sources for it from xiph.org.
If I'm wrong here I'll rectify it.
[1] Well, I changed the version number. Whoo!
FLAC libraries are BSD licensed.
FLAC frontend is GPL. The frontend has no changes[1]. Does linking the GPL frontend with the BSD libraries force you to publish the source for the libraries, or just for the frontend?
[a href="index.php?act=findpost&pid=357614"][{POST_SNAPBACK}][/a]
Define linking. If you link the objects together creating one binary/library - yes GPL forces you to publish verything which is linked, even if original sources were BSD. If you link "hard-dynamic" ie, your exe won't start using the lib, but both parts are locates in different part (one exe, one lib), you are in a legal grey area, as GPL still wants you to open source of linking exe (well, check whether there are GPL exeptions), but you could argue that you didn't modify anything of the GUI app and infact could supply the lib w/o the GUI. So best would be just supplying libs/ exe w/o GPL components involved, then you stay out of legal trouble.
Nevertheless, I wouldn't call it very polite to take a open source app and enhance it or do whatevery with it w/o giving anything back to the project.
IANAL, but I was under the impression that combined works have to be under the GPL.
http://www.gnu.org/licenses/gpl.html (http://www.gnu.org/licenses/gpl.html)
http://www.gnu.org/licenses/gpl-faq.html#WhatDoesCompatMean (http://www.gnu.org/licenses/gpl-faq.html#WhatDoesCompatMean)
Okay, I've removed the binary. I'll talk to Josh about how to make the change in a way that doesn't cause the library to stop being threadsafe
Nevertheless, I wouldn't call it very polite to take a open source app and enhance it or do whatevery with it w/o giving anything back to the project.
You are right. It's more polite not to enhance open source software at all. I won't do it anymore, I promise!
Is there a way to get the modified FLAC encoder to work with FlacDrop or Speek's FLAC frontend? Doesn't work for me...
Why don't you want to release the source code anyway? any particular reason?
It's to annoy the open source zealots [1].
[1] That, and the fix being suboptimal, probably not threadsafe, and really crummily written so I find it embarassing. I'd rather have Josh fix it proper, thankyouverymuch.
Okay, I've removed the binary. I'll talk to Josh about how to make the change in a way that doesn't cause the library to stop being threadsafe
Nevertheless, I wouldn't call it very polite to take a open source app and enhance it or do whatevery with it w/o giving anything back to the project.
You are right. It's more polite not to enhance open source software at all. I won't do it anymore, I promise!
[a href="index.php?act=findpost&pid=357638"][{POST_SNAPBACK}][/a]
It is greatly appreciated to see these improvements. I trust the majority of the community is greatful and will encorage such efforts to continue.
Okay, I've removed the binary. I'll talk to Josh about how to make the change in a way that doesn't cause the library to stop being threadsafe
Nevertheless, I wouldn't call it very polite to take a open source app and enhance it or do whatevery with it w/o giving anything back to the project.
You are right. It's more polite not to enhance open source software at all. I won't do it anymore, I promise!
[a href="index.php?act=findpost&pid=357638"][{POST_SNAPBACK}][/a]
Yeah, you already showed in the FAAD debate you don't understand much about GPL and its ideas, so how can you possibly understand that you don't help the project by just optimizing it for yourself to show off?
Of course, if you're mailing the mantainer the ideas or patches to let it be done in a proper way, then its a different story...
Yeah, you already showed in the FAAD debate you don't understand much about GPL and its ideas, so how can you possibly understand that you don't help the project by just optimizing it for yourself to show off?
Of course, if you're mailing the mantainer the ideas or patches to let it be done in a proper way, then its a different story...
[a href="index.php?act=findpost&pid=357658"][{POST_SNAPBACK}][/a]
...which he said he's doing.
Garf,
I do not want to get embroiled into this too much especially because I admit that I do not understand GPL 100%. But you have to get used to the fact that some of us including myself runs only Linux. There is no zealotry here on my part, but we cannot even try your work. (Besides due to virus threat I would not run any executable. I would only accept a patch.)
In addition there is a courtesy that if you change a GPL code, esp if you start a thread on that to invite discussion, then you provide a source code in the spirit of the original code - even with caveats - for peer review or for just a good-will. Without this it looks like a show-off and in some ways tarnish your very good reputation. This is just how things work in our minds. Pointless to talk about legal language here. I am a DSP engineer myself in a different field, and within my company I am always very open about my work, because I know that other behavior would poison the teamwork. FLAC is in a way our community project in a similar way.
Triza
Okay, I've removed the binary. I'll talk to Josh about how to make the change in a way that doesn't cause the library to stop being threadsafe
[a href="index.php?act=findpost&pid=357638"][{POST_SNAPBACK}][/a]
People who still have the binary have the right to ask for the source.
Just a quick question to the GPL experts from the GPL dummy (myself)
If I got everything right - FLAC itself is using:
BSD Library (FLAC codec) + GPL Frontend (command line FLAC executable)
Is this correct?
If it is, then Garf changed the BSD Library part... but people demanded source because he actually published FLAC GPL frontend (which, itself is using BSD library, right?)...
I don't get some things here - here are my questions (I am a complete dummy here, so I just want to get myself educated with these questions):
a) Can GPL work legally contain work licensed under other licenses inside?
b) If a) is "yes" - can they be linked together in the single .exe file?
c) If a) is "yes", does entire package then becomes GPL licensed?
- If yes... it is a big legal problem IMO, since you just changed license of the package? How can this be done?
- If no... then how can be entire package treated as the GPL?
Many thanks in advance for sheding some light here - it will be a great help for my understanding of the GPL.
>a) Can GPL work legally contain work licensed under other licenses inside?
Yes and it does (with operating system dependancies and compiler libraries).
Linking a BSD libarary to a GPL Frontend does NOT make the BSD library GPL, but there is an obligation to release a package that can be built into the GPL front end (a static lib...)
As much as I dislike Garf's not releasing the source (even it is it sub_something, it is still useful to look at and base things on, even Josh asked to see it.), he has no GPL requirement to release it, but perhaps a moral one
I'd say yes, yes, and yes.
In section 2 of the GPL, it says:
These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.
libflac is part of flac_msvc.exe which is a work based on flac.c, the distribution of flac_msvc must be on the terms of the GPL, whose permissions for me extend to the entire flac_msvc.exe, and thus even to the originaly-BSD libraries.
Linking a BSD libarary to a GPL Frontend does NOT make the BSD library GPL, but there is an obligation to release a package that can be built into the GPL front end (a static lib...)
[a href="index.php?act=findpost&pid=357747"][{POST_SNAPBACK}][/a]
That seems to contradict what kjoonlee wrote, which seems to say that the GPL would infect the BSD part too, meaning providing it as a .lib wouldn't be sufficient.
I suppose providing separate libflac DLLs would have been safe, but I don't think there's a way out if you use a static binary.
Yeah, you already showed in the FAAD debate you don't understand much about GPL and its ideas, so how can you possibly understand that you don't help the project by just optimizing it for yourself to show off?
[a href="index.php?act=findpost&pid=357658"][{POST_SNAPBACK}][/a]
People can use the binary and get more tightly compressed FLAC's.
This is how the
entire non-free software world works, and you know what, it works fine too!
In addition there is a courtesy that if you change a GPL code,
I did not change GPL code.
I suppose providing separate libflac DLLs would have been safe,
[a href="index.php?act=findpost&pid=357751"][{POST_SNAPBACK}][/a]
That seems to be false, too. At least, SRC is a famous example. It came up very recently in the foobar2000 forum again.
GPL cannot change the license of existing items, the Flac lib (BSD) came before the flac front end (GPL), ie the flac front end uses the flac lib, they are two are seperate entities.
Even within GPL its self, using an existing GPL v1 component with a GPL v2 project does not make the first component suddenly GPL v2, it stays v1.
Edit:
Nowhere in the GPL does it say all items of source must be GPL when using existing items. For example here is some code:
//--------------------------------
// Released as BSD, bla bla
//--------------------------------
int MultiplyBy2 (int v)
{
return(v * 2);
}
It is possible to use that source file within a GPL project as BSD is compatible with GPL, that source remains BSD. If garf comes along and improves the sub (by making it faster: ie #define MultiplyBy2 (v) (v*2)) it can stay BSD, there is no need to release that code, but if garf was to release a GPL front end that used it then he has an obligation to release something that will still compile (think of it as a fork, if it suddendly became windows only, or for a specific compiler). Nothing has been taken away from the GPL, after all you can either build the old front end, or build the new front end (even though all the source is not present).
GPL cannot change the license of existing items, the Flac lib (BSD) came before the flac front end (GPL), ie the flac front end uses the flac lib, they are two are seperate entities.
Even within GPL its self, using an existing GPL v1 component with a GPL v2 project does not make the first component suddenly GPL v2, it stays v1.
[a href="index.php?act=findpost&pid=357755"][{POST_SNAPBACK}][/a]
The GPL can change the license of existing items. Let's say there's a public-domain library. I can rerelease it under the GPL, with a GPL program that I wrote.
The newer BSD library allows this, AFAIK.
The GPL can change the license of existing items. Let's say there's a public-domain library. I can rerelease it under the GPL, with a GPL program that I wrote.
Public domain is "no license" - you can completely claim ownership to it, as the part of large works.
BSD library has a copyright, and relevant copyright holders - I am not sure if that allows changing of the license by some third party?
Could you please pinpoint me to the part of BSD license which states that users of the BSD works could change the license conditions?
>The GPL can change the license of existing items...The newer BSD library allows this, AFAIK
Only if you were to take the existing code, and rebrand it as GPL (if compatible with the existing license, ie at the top of the code put GPL stuff), and then this can only be done (in my opinion) if the code has changed in the GPL derrived stuff, putting GPL at the top does not change the work, only the supposed labeling - I can remove the label and hey presto it is identical to the BSD item, hence it still is BSD.
FLAC has the items (lib and front end) as seperate source trees.
Edit: I would also hazzard a guess that the official FLAC front end for windows uses an import library from the FLAC Library to build, ie taking an existing component and blending to the final front end...
The GPL can change the license of existing items. Let's say there's a public-domain library. I can rerelease it under the GPL, with a GPL program that I wrote.
Public domain is "no license" - you can completely claim ownership to it, as the part of large works.
BSD library has a copyright, and relevant copyright holders - I am not sure if that allows changing of the license by some third party?
Could you please pinpoint me to the part of BSD license which states that users of the BSD works could change the license conditions?
[{POST_SNAPBACK}][/a]
(http://index.php?act=findpost&pid=357758")
IANAL. [a href="http://www.gnu.org/licenses/gpl-faq.html#OrigBSD]http://www.gnu.org/licenses/gpl-faq.html#OrigBSD[/url]
This seems to imply that the old BSD license does not allow this, but the new BSD (which Xiph project libraries tend to use) does.
>The GPL can change the license of existing items...The newer BSD library allows this, AFAIK
Only if you were to take the existing code<snip>
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=357759")
What do you think about [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=40452&view=findpost&p=357748]post #48[/url] above?
http://www.kallisys.org/bsd-lite/bsd-gpl/?lg=en (http://www.kallisys.org/bsd-lite/bsd-gpl/?lg=en)
Combining BSD and GPL generates a big confusion - because of the requirement of GPL that whole GPL work needs to be published, even parts of the work that do not impose such restrictions.
Very interesting:
One clever objection that can be made of this (thanks David) is that the BSD license only covers some module you might want to include in your work. License (including conditions) has to be kept with these modules, not with the whole code.
I'm not sure that the GPL allows this because of section 2 which seems to explicitly say that if you release the whole source code for a program (basically what you're required to do when asked if it's released under the GPL), the whole and parts are licensed under the GPL:
These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.
But even if this was allowed by the GPL (and by the BSD, i.e. the whole thing isn't considered as the original BSD-licensed code plus modifications), there is a problem with the binary form. Clause #2 of the BSD says that the documentation coming with the binary form (or other material) should include the BSD license. The binary form will include both the original BSD-licensed code and the GPL code. You cannot distinguish both parts and provide a license for each part.
Funny, eh?
In addition there is a courtesy that if you change a GPL code,
I did not change GPL code.
[a href="index.php?act=findpost&pid=357753"][{POST_SNAPBACK}][/a]
Secion 0 of the GPL has a part that says even verbatim copying counts.
0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you".<snip>
So, what - besides licenses - is the reason *not* to release the sources?
>What do you think about post #48 above
I think it is miss-interpreted (and very ambiguitous, the wording is so full of legal holes you could sail an oil tanker through most of the GPL pre-amble, lawyers would have a field day in court), GPL was not designed to take peoples rights away, derriving something from the public domain, I would say would keep those items in the public domain (by making those public domain items GPL you are removing rights and limiting people, not what GPL of all things are about). GPL was written to stop certain circumstances (taking something that is clearly GPL and making it non-GPL and restrictive), this is not one of them.
So, what - besides licenses - is the reason *not* to release the sources?
[a href="index.php?act=findpost&pid=357767"][{POST_SNAPBACK}][/a]
I'd be very interested in the answer too, thanks.
Also, has Josh been made aware of the changes/rough optimizations in the meantime please?
EDIT: Grammar.
Asking question that are answered one page up is not smart.
Asking question that are answered one page up is not smart.
[a href="index.php?act=findpost&pid=357774"][{POST_SNAPBACK}][/a]
And replying like this is pretty rude while serving only to underline your lack of class, thanks anyway.
EDIT: Class part.
Can you just clarify my understanding of the situation here.
- Garf spots a potential bug, or oversight, in Josh's FLAC code and improves it
- A few people thank Garf, a few people start demanding source code
- Continued discussion about licensing
- Garf removes binary and says he'll simply tell Josh where the fix was, as he is worried about his code
- Continued discussion about licensing, and further demands for Garf's code (which he believes is unsafe and suboptimal)
- Continued discussion about licensing
Garf, you insideous creature. I demand that you hand over not only the source code, but the shirt you were wearing while you wrote it! It is rightfully mine under the FKU2 licensing agreement (link to come).
I just find it all quite demoralising, s'all. It seems people are more interested in the code being available, than whether it actually works, and whether the author should be thanked and/or treated with some respect. This is not the first time I have seen such threads.
Source code:
http://sjeng.org/ftp/lossless/flac-1.1.2.1.rar (http://sjeng.org/ftp/lossless/flac-1.1.2.1.rar)
I just find it all quite demoralising, s'all. It seems people are more interested in the code being available, than whether it actually works, and whether the author should be thanked and/or treated with some respect. This is not the first time I have seen such threads.
[a href="index.php?act=findpost&pid=357781"][{POST_SNAPBACK}][/a]
The thing is, as a programmer, you don't like releasing code that is known to be 99% sure buggy. On the other hand, you want people to test it, so you can axe the bugs. I thought it was safe to do this with libFLAC since it's BSD licensed, but indeed you would have to write your own frontend then. I'm doing this entirely in for free in my own spare time, and I don't even use FLAC myself (WavPack!), so writing another frontend is out of the question.
I can understand people on Linux want to have it too, but that's really better done by Josh integrating a correct and fully working version of this, rather than have people sitting around with this kind of hack. I don't think this compiles on Linux, for starters.
Now, if distributing that flac encoder binary is a violation of FLAC's license, then I'll quit doing it, obviously! No need to start a holy war for that. I already said in post 34 that I would rectify it if needed, and I just did. The source is up there, and I'm sure there is much more room for improvement.
It seems that all the GTune encoders were illegal too, since although libvorbis is LGPL, the vorbis frontends are GPL, I think. I also didn't publish source there until I was reasonably pleased with it.
Garf, first of all, thank you for spotting the bugs and doing something about it, and thank you for releasing the source.
PS. libvorbis is now BSD as well. Since when, I'm not sure, but it happened later than beta 4.
I just find it all quite demoralising, s'all. It seems people are more interested in the code being available, than whether it actually works, and whether the author should be thanked and/or treated with some respect. This is not the first time I have seen such threads.
[a href="index.php?act=findpost&pid=357781"][{POST_SNAPBACK}][/a]
The thing is, as a programmer, you don't like releasing code that is known to be 99% sure buggy. On the other hand, you want people to test it, so you can axe the bugs. I thought it was safe to do this with libFLAC since it's BSD licensed, but indeed you would have to write your own frontend then.
It seems that all the GTune encoders were illegal too, since although libvorbis is LGPL, the vorbis frontends are GPL, I think. I also didn't publish source there until I was reasonably pleased with it.
[a href="index.php?act=findpost&pid=357788"][{POST_SNAPBACK}][/a]
As I said, it would be perfectly OK, if you just released the library, wo any GPL involved code. Nobody forces anyone to use the GPLed frontend with it. You are not forced to write a non-GPL front-end either. So then nobobdy could demand anything from you. It is up to the user to decide to use a GPLed prog with a close-sourced lib. Then it is a problem for the front-end / user and not for you, as the user uses a GPL prog with a license-incompatible lib.
The situation would be different if you write a closed-sourced app to use a GPL lib (ie the other way round as the situation here). Even if you don't supply the GPL lib in binary form, this is strictly not allowed.
But here the situation with flac/ogg vorbis is rather awkward, but not forbidden - if you closely take care of what components you supply/distribute.
(BTW, as being a hobby dev myself I understand that one doesn't want to immediately publish experimantal/broken code. But first you simply ignored the calls for sources, so this wasn't promising. But I appreciate that you have given out the sources, in the end.)
PS. libvorbis is now BSD as well. Since when, I'm not sure, but it happened later than beta 4.
[a href="index.php?act=findpost&pid=357809"][{POST_SNAPBACK}][/a]
It doesn't matter. libFLAC is BSD, and see what happened.
The problem is the frontends.
ok, Garf and I are corresponding.
since Garf posted the code some of this is kind of moot. but what started this whole debate (providing only binaries for flac when only libFLAC code changed) does not really go against the intentions of the licensing I chose. the community reaction was more to a perceived 'etiquette breach' and second-guessing what Garf was going to do with his changes. I think according to the GPL I could have demanded the code, but notice that I didn't and I wouldn't in this case. as much as I would like to enforce that all libFLAC changes get contributed back, I went to BSD to "let it go" in the interest of better adoption. competing implementations I think are a healthy sign for a codec (as long as they are compliant). partly because of that, FLAC as far as I know is the only lossless codec with multiple independent implementations.
the other lesson to take from this is that distributing GPL binaries can be tricky. I will go back over the LGPL to see if that more closely matches my intent for flac/metaflac.
Josh
Forget it. Whatever I wrote is obsolete.
the other lesson to take from this is that distributing GPL binaries can be tricky. I will go back over the LGPL to see if that more closely matches my intent for flac/metaflac.
Josh
[a href="index.php?act=findpost&pid=357851"][{POST_SNAPBACK}][/a]
Being pedantic, you could also get in trouble doing this. Why? Because you probably incorporated other people's code into the GPL components. So by default, you have to assume these people gave you the code as GPL and that code - though merged in your repository is strictly still their code (unless they explicitly stated otherwise, of course). You you cannot simply relicense it - esp to a more liberal license. In fact you would have to ask every one who contributed, whether they are fine with the license change...
the onus in GPL is on the distributor and the FLAC project releases complete source code always.
of course this must be carefully considered. I can't relicense someone else's code. the parts of flac/metaflac aside from FLAC project code are replaygain_analysis (LGPL), replaygain_synthesis (GPL by our own John Edwards) and some charset/string code (mix of GPL and BSD).
Josh
I think I followed this thread...and all acronyms aside...I'm excited to see what becomes of the modified code...I'm a big fan of flac and as a user would like to see -6, -7,and -8 squeak out a better ratio.
This is simply my very non-educated super noob $0.02...
It may be non-educated, but to me it is the most sensible response.
It's nice to hear someone being positive about a potential benefit to the FLAC community.
I hope that Garf is correct in his theory, and that Josh can find a way to implement it cross-platform.
Well I too am pleased and excited to see such potential improvements to a codec I use for my backup purposes. I think what Garf has done is actually very kind - the work may not have started out with the intention of being a "kind" thing to do, but imo it's resulted in that. Garf looked at the code, found an area in which to enhance and produce a better result, gave it to us for testing (altho not for *nix) and now Josh presumably has the code changes to see what he makes of them.
All in all, good stuff. Nice to see tbh
I agree in that Garf rocks for helping FLAC improve, and the patent and source debate is secondary.
I do think it's a good thing to release the source, even if it's ugly, a hack and most certainly filled with bugs, it might inspire other people to rewrite it and speed up it's inclusion in the FLAC project.
Interesting read....the more activity the better.
Is there a windows compile of Garf´s version around somewhere?
I came too late on this topic to get the improved encoder at time. Now it's available again (link on first post), I performed some tests.
I must say that I'm pleasantly surprised by the level of improvement. I only expected one or two saved kbps, but in fact I often get a bitrate reduction superior to 10 kbps. For many people such performance may appear as totally ridiculous (and it is) but I know that such improvements are hard to get when you don't sacrify the decoding speed.
On 150 classical music tracks I obtained (in summary):
•
default mode [-5]flac 1.1.2 -5 637,81 kbps
flac 1.1.2.1 -5 630,91 kbps
=> 7 kbps without any compromise on decoding speed
•
strongest but usable mode [-best or -8]flac 1.1.2 -8 634,97 kbps
flac 1.1.2.1 -8 625,11 kbps
=> 10 additional kbps this time!!
This new step forward is making FLAC much more competitive compared to WavPack. At
~comparable settings, I obtained:
•
efficient encoding speed modeWavPack 4.31 -f 655,57 kbps
WavPack 4.31 [defaut] 645,26 kbps
flac 1.1.2.1 -5 630,91 kbps
•
strongest ratio modeWavPack 4.31 -fx5 628,72 kbps
WavPack 4.31 -x4 618,27 kbps
flac 1.1.2.1 -8 625,11 kbps
In other words, I would now consider FLAC as clearly more efficient with classical music with default and "not-strong-asymetrical" mode. With strong-asymetrical mode, flac provided a slightly better compression efficiency than WavPack on average for the same decoding speed (i.e. WavPack fast) but a worse one compared to the normal encoding mode of WavPack (which requires a bit more CPU cycles for decoding than flac).
Furthemore, WavPack has strong efficiency issue with some mono tracks (still unsolved). Bryant, WavPack is needing you...
Last comparison:
FLAC vs ALAC which were previously close on compression side:
flac 1.1.2.1 -5 630,91 kbps
flac 1.1.2.1 -8 625,11 kbps
ALAC iTunes 655,45 kbps
ALAC dBPowerAmp 658,09 kbps
Nice work, Garf!
N.B. The tested samples (150 full tracks selected for representativity of my library) offers different result as my previous material (20 complete albums) (http://guruboolez.free.fr/lossless). In the old one, WavPack -f and FLAC -5 were on the same level (611 kbps) but on this one official FLAC is already 17 kbps more efficient than WavPack -f !!!
Moreover, with foobar2000 0.9, FLAC is very slightly faster on decoding stage whereas foobar2000 0.83 shown the opposite. The funniest is that decoding speed has significantly increased for both formats (x~45 -> x~60 on the same computer! Wow, that's speed optimisation [of foobar2000 input components] )
P.S. It seems that flac 1.1.2.1 is slower on encoding side than reference FLAC. Could someone confirm it? I haven't properly measured the encoding speed (because I was working on my computer during the encoding process).
P.S.(2): I can post the complete bitrate table on demand (if someone is interested by it).
P.S. It seems that flac 1.1.2.1 is slower on encoding side than reference FLAC. Could someone confirm it? I haven't properly measured the encoding speed (because I was working on my computer during the encoding process)
It's slightly slower (about 6-10% IIRC). It's not caused by the changes I made (if anything, it should be faster), but it seem FLAC is very sensitive to the used compiler settings, and I didn't spend too much time finding the right combination.
P.S. It seems that flac 1.1.2.1 is slower on encoding side than reference FLAC. Could someone confirm it? I haven't properly measured the encoding speed (because I was working on my computer during the encoding process)
It's slightly slower (about 6-10% IIRC). It's not caused by the changes I made (if anything, it should be faster), but it seem FLAC is very sensitive to the used compiler settings, and I didn't spend too much time finding the right combination.
[a href="index.php?act=findpost&pid=377957"][{POST_SNAPBACK}][/a]
I can also confirm it is slower on an A64 system.
btw. i encoded a bunch of CDs and can say it works without problems on a Squeezebox3. So one hardwareplayer seems 1.1.2.1 approved
Does anyone know a flac->flac transcoding tool for the same directory and remembers the tags?
This new step forward is making FLAC much more competitive compared to WavPack. At ~comparable settings, I obtained:
• efficient encoding speed mode
WavPack 4.31 -f 655,57 kbps
WavPack 4.31 [defaut] 645,26 kbps
flac 1.1.2.1 -5 630,91 kbps
• strongest ratio mode
WavPack 4.31 -fx5 628,72 kbps
WavPack 4.31 -x4 618,27 kbps
flac 1.1.2.1 -8 625,11 kbps
[a href="index.php?act=findpost&pid=377951"][{POST_SNAPBACK}][/a]
I noticed you didn't test WavPack high mode (-h). It should give better compression than the -x modes.
Does anyone know a flac->flac transcoding tool for the same directory and remembers the tags?
I'd be interested in this, too. One, if they were resourceful enough, could probably write a script to emulate this function.
I noticed you didn't test WavPack high mode (-h). It should give better compression than the -x modes.
[a href="index.php?act=findpost&pid=378103"][{POST_SNAPBACK}][/a]
True, but as I said, I performed a comparison at
~comparable settings. -high mode compress indeed better at a very fast speed, but the decoding speed is by far slower than FLAC. IIRC, Bryant said that iPod couldn't decode -h encoding on RockBox devices. Comparison with FLAC wouldn't be really fair IMO (if you take the decoding speed as the main criterion).
P.S. It seems that flac 1.1.2.1 is slower on encoding side than reference FLAC. Could someone confirm it? I haven't properly measured the encoding speed (because I was working on my computer during the encoding process)
It's slightly slower (about 6-10% IIRC). It's not caused by the changes I made (if anything, it should be faster), but it seem FLAC is very sensitive to the used compiler settings, and I didn't spend too much time finding the right combination.
[a href="index.php?act=findpost&pid=377957"][{POST_SNAPBACK}][/a]
I can also confirm it is slower on an A64 system.
btw. i encoded a bunch of CDs and can say it works without problems on a Squeezebox3. So one hardwareplayer seems 1.1.2.1 approved
Does anyone know a flac->flac transcoding tool for the same directory and remembers the tags?
[a href="index.php?act=findpost&pid=377959"][{POST_SNAPBACK}][/a]
Maybe you should try foobar2000.
foobar2000 requires to output to a different name and to manually delete the source files (then eventually to rename the new ones). Unless there's a new feature in 0.9...
P.S. It seems that flac 1.1.2.1 is slower on encoding side than reference FLAC. Could someone confirm it? I haven't properly measured the encoding speed (because I was working on my computer during the encoding process).
This is my experience. Please note that I used the first binary originally compiled at the beginning of the thread. (PC is a 1560mhz Intel Celeron Tualatin, samples set is heterogeneus).
1.1.2 -0 38,08s 33,64s 938kbps
1.1.2.1 -0 34,79s 27,05s 938kbps
1.1.2 -1 40,17s 34,81s 924kbps
1.1.2.1 -1 35,83s 28,68s 924kbps
1.1.2 -2 48,62s 34,71s 921kbps
1.1.2.1 -2 44,00s 28,58s 921kbps
1.1.2 -3 57,31s 36,05s 890kbps
1.1.2.1 -3 58,01s 30,14s 881kbps
1.1.2 -4 78,62s 37,48s 867kbps
1.1.2.1 -4 79,09s 31,95s 858kbps
1.1.2 -5 99,00s 37,18s 866kbps
1.1.2.1 -5 100,13s 31,93s 857kbps
1.1.2 -6 104,80s 36,95s 866kbps
1.1.2.1 -6 106,77s 30,90s 856kbps
1.1.2 -7 287,60s 38,03s 865kbps
1.1.2.1 -7 253,25s 32,03s 854kbps
1.1.2 -8 372,08s 38,65s 862kbps
1.1.2.1 -8 321,89s 32,13s 848kbps
1.1.2 -8(-l30 -r0,8) 1178,87s 41,03s 860kbps
1.1.2.1 -8.1(-l30 -r0,8) 959,86s 34,80s 845kbps
Improvements are as follow (enc/dec/size):
0 8,64% 19,59% 0,00%
1 10,80% 17,61% 0,00%
2 9,50% 17,66% 0,00%
3 -1,22% 16,39% 1,01%
4 -0,60% 14,75% 1,04%
5 -1,14% 14,12% 1,04%
6 -1,85% 16,37% 1,15%
7 11,94% 15,78% 1,27%
8 13,49% 16,87% 1,62%
-l30 -r0,8 18,58% 15,18% 1,74%
AVERAGE 6,81% 16,43% 0,89%
Decoding speed improvement and file size improvement look very interesting to me, as well as encoding speed at -6 and over. I don't have any explanation for encoding speed being so fluctuating.
Does anyone know a flac->flac transcoding tool for the same directory and remembers the tags?
I'd be interested in this, too. One, if they were resourceful enough, could probably write a script to emulate this function.
[a href="index.php?act=findpost&pid=378139"][{POST_SNAPBACK}][/a]
On linux I use this script:
-----
for file in *.flac
do
# export tags, convert to wav and delete the flac
metaflac --export-tags-to="$file".txt "$file"
flac -d --delete-input-file "$file"
wait
# encode the wav to flac and delete the wav
for fiwav in *.wav
do
nice -n 5 flac -V -8 --delete-input-file "$fiwav"
wait
done
# import tags and delete the txt file
metaflac --import-tags-from="$file".txt "$file"
rm -f "$file".txt
done
sleep 5
# add replygain
metaflac --add-replay-gain "$file"
------
the other lesson to take from this is that distributing GPL binaries can be tricky. I will go back over the LGPL to see if that more closely matches my intent for flac/metaflac.
Josh
[a href="index.php?act=findpost&pid=357851"][{POST_SNAPBACK}][/a]
IANAL but there appears to be much confusion here and I've some experience in sorting such confusion out.
Firstly it is always upto the copyright holder to decide what license something is distributed under. However some licenses allow additional conditions to be attached.
I can take BSD licensed code and release it under a different license as long as I follow the conditions of the the license and reproduce the copyright notice. What I catagorically can not do is remove that copyright notice.
The GPL does not allow additional conditions to be attached so I cannot take GPLed code and release it under a different license. The GPL is primarily about distribution, I can do whatever I like with the binaries including sell them but I must make an offer of the source to anyone in posession of the binary. I can not impose additional conditions over and above the GPL on the person who receives that source. They are therefore free to distribute that source or compile and give away binaries, again making an offer of the source to anyone they give a binary to.
The GPL does not really differentiate between dynamic and static linking however the difference is crucial when dealing with mixed licenses, this is where people get very confused about the GPL "infecting" code under other (GPL compatable) licenses.
In this case FLAC frontend is distributed under the GPL and libflac under BSD(xiph) license. Now lets assume I
- Make no modifications to FLAC frontend
- Make some modifications to libFlac which I do not want to release the source for.
If I statically link the two together I cannot distribute the resultant binary at all due to section 2b of the GPL
"You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License."
Even if I dynamically link FLAC Frontend to libFlac I still get caught under this clause if the two are distributed together as a bundle. This would not come under the mere aggregation clause of the GPL.
Now if I
ONLY distribute a modified libFlac in binary form with no Flac Frontend I am perfectly OK since the Xiph(BSD) license that it is under allows me to do this.
A recipient of this modified libFlac may, themselves, build an executable using source code for FLAC Frontend and this binary library however they would also
NOT be allowed to distribute such a binary for the same reasons as before.
There is a vast amount of complication if you start going down the road of attempting to explicitly dynamlically load modules such as kernel modules, dynamic libraries conforming to plugin API using LoadLibrary/dlopen etc. which are licensed under a GPL incompatible license into a GPLed program. The GPL is very vague on such matters.
Does anyone know a flac->flac transcoding tool for the same directory and remembers the tags?
I'd be interested in this, too. One, if they were resourceful enough, could probably write a script to emulate this function.
[{POST_SNAPBACK}][/a]
(http://index.php?act=findpost&pid=378139")
I’ve made a pretty extensive batch script for FLAC conversion.
It will do:
Flac to flac
Flac to mp3
Flac to ogg
Flac to aac
Flac to wma
You can download it here: [a href="http://www.revnull.com/download.php?file=!convert.zip]http://www.revnull.com/download.php?file=!convert.zip[/url]
(Cut and paste the URL into your browser or the link will not work.)
If you have any questions about it, email me at rramirez@revnull.com
- Ray
Does anyone know a flac->flac transcoding tool for the same directory and remembers the tags?
I'd be interested in this, too. One, if they were resourceful enough, could probably write a script to emulate this function.
[{POST_SNAPBACK}][/a]
(http://index.php?act=findpost&pid=378139")
I’ve made a pretty extensive batch script for FLAC conversion.
It will do:
Flac to flac
Flac to mp3
Flac to ogg
Flac to aac
Flac to wma
You can download it here: [a href="http://www.revnull.com/download.php?file=!convert.zip]http://www.revnull.com/download.php?file=!convert.zip[/url]
(Cut and paste the URL into your browser or the link will not work.)
If you have any questions about it, email me at rramirez@revnull.com
- Ray
[a href="index.php?act=findpost&pid=378199"][{POST_SNAPBACK}][/a]
This works really cute Thanks. One small problem with the Tags. Date is the actual date. This may be cause my Flacs most of the time have no entry there.
I've never encountered that problem before. I always make sure my tags are complete before I run a batch conversion. A great program for tag editing/completion is mp3tag (http://www.mp3tag.de/en/index.html). Best of all, it's free.
- Ray
I've never encountered that problem before. I always make sure my tags are complete before I run a batch conversion. A great program for tag editing/completion is mp3tag (http://www.mp3tag.de/en/index.html). Best of all, it's free.
- Ray
[a href="index.php?act=findpost&pid=378240"][{POST_SNAPBACK}][/a]
I know.
I am currently at letter C... Looks like the reencoding will take a while.
Many thanks again for your batch. I only added -V to the encoding options.
One result for --best:
1.1.2 148,1GB ~829kbps
1.1.2.1 146,5GB ~820kbps
a little progress report: thanks to Garf's suggestions I have been doing more experiments and can consistently get 1-2% (depending on how you measure) compression improvements with no change to the decoder and no decrease in decoding speed. so this will definitely be in the next version of FLAC in august/september. I may upload an intermediate version here soon to enlist help with re-tuning the presets (-0 to -9).
Josh
1-2% compared to 1.1.2 or to 1.1.2.1? Because 15 kbps in addition to the ~10 kbps offered by 1.1.2.1 would make FLAC very competitive with more complex formats (like Monkey's -c1000/-c2000).
sorry, I meant compared to 1.1.2. the last test I ran (still more work to do) on the corpus here (http://flac.sourceforge.net/comparison.html) got 407.65MB for flac -5 and 405.21MB for flac -8.
Josh
new encoder is here:
http://www.hydrogenaudio.org/forums/index....showtopic=44229 (http://www.hydrogenaudio.org/forums/index.php?showtopic=44229)
I just tried 1.1.2.1 and found some interesting results...
64,052,060 01 - Cyndi Lauper - Girls Just Want[12'].wav
41,799,092 01 - Cyndi Lauper - Girls Just Want[12'].flac <-- 1.1.2_CVS
41,845,296 01 - Cyndi Lauper - Girls Just Want[12'].flac <--- 1.1.2.1
42,002,097 01b - Cyndi Lauper - Girls Just Want[12'].flac <--- 1.1.2
41,389,602 01 - Cyndi Lauper - Girls Just Want[12'].wv
The command lines used are as follows
FLAC 1.1.2
-8 -V -b 4096 --sector-align
FLAC 1.1.2_CVS
-8 -A tukey(0.25) -A gauss(0.1875) -V -b 4096 --sector-align
FLAC 1.1.2.1
-8 -V -b 4096 --sector-align
Wavpack
-x3 -h
So, what's interesting here? It's no wonder that flac-cvs compresses better than 1.1.2.1.