HydrogenAudio

Lossless Audio Compression => FLAC => Topic started by: GeorgeFP on 2009-11-15 18:38:43

Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-15 18:38:43
Hi!

Maybe you already know fpMP3Enc (http://www.hydrogenaudio.org/forums/index.php?showtopic=73790), my multicore MP3 encoder...

This time, I'd like to introduce my multicore FLAC encoder. You can download the binaries here (http://www.thinkmeta.de/download/fpflac_beta.7z).

The package contains Win32 binaries for XP, Vista and 7, and Win64 binaries for Vista and 7.

The FLAC encoder is part of my "fpStream" project. The command-line to convert a WAV file to FLAC is as follows:

Code: [Select]
FPSTREAM.EXE readfile in -f <wav-filename> + wavin wav -s in + fpflacenc enc -s wav [-<compression level>] + writefile out -s enc -f <flac-filename>


The benchmarks so far:
I used the same test suite as in fpMP3Enc (Intel Q9450, 61 WAV files, about 5 hours playing time). The compression level was 5. The files were encoded one after the other, not in parallel.

- flac 1.2.1: 3 min 26 secs
- fpFLAC x64: 1 min 13 secs

So, the speedup was 2.8x. This is quite good if you consider that this version does not contain any SSE optimizations.

What's missing:
- SSE optimizations
- Metadata support (tags, seek points etc.)
- free of bugs

And some limitations:
- Win32 version: The total WAV file size sum must not exceed 400MiB(? - not sure).
- Win64 version: The total WAV file size sum must not exceed 400GiB.
- Input: WAV 16-bit stereo, 44.1kHz

Comments and feedback are welcome!
George
Title: A multicore FLAC encoder
Post by: Cokemonkey11 on 2009-11-15 20:49:35
Good stuff, can't wait for the GUI.
Title: A multicore FLAC encoder
Post by: punkrockdude on 2009-11-15 22:03:49
Yeah! I love that your make multi core applications out of stuff I use regularly. I too can't wait for a GUI or a version that can accept the standard FLAC commands. Regards
Title: A multicore FLAC encoder
Post by: flacflac on 2009-11-16 10:14:02
Wow, this is very interesting.  The speed-gain you're describing happens when just converting a single file?  Is there any danger of disk-thrashing?  How many threads are being calculated at once?
And lastly:  is this an open source tool?

Thank you for this development.
Title: A multicore FLAC encoder
Post by: Maurits on 2009-11-16 11:39:27
Is there any chance that these improvements can be used 'upstream' in the original FLAC sources? That way the benefits of this are not limited just to the users of one specific operating system.
Title: A multicore FLAC encoder
Post by: itisljar on 2009-11-16 11:55:53
It would be interesting to compare it to CUDA FLAC.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-16 12:34:55
Wow, this is very interesting.  The speed-gain you're describing happens when just converting a single file?  Is there any danger of disk-thrashing?  How many threads are being calculated at once?
And lastly:  is this an open source tool?

Thank you for this development.

This evening I will provide a new package which will also contain SSE optimized versions. The SSE4 version made the test suite in 55s which is a speedup of 3.75x.

The I/O performance was 72.7 MiB/s (3 GiB read, 1 GiB write) when converting single files.

The whole process is stream-based, so there shouldn't be any disk-thrashing.

About the number of threads on a quad-core system:
- 1 main thread for the application (idle most of the time)
- 4 CPU worker threads that do the encoding
- 1 I/O thread for asynchronous I/O
- 1 I/O completion port thread

So, the formula is: numThreads = 3 + numCores

About open source: Yes, it's open source. The code will be available for download soon.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-16 12:56:15
Is there any chance that these improvements can be used 'upstream' in the original FLAC sources? That way the benefits of this are not limited just to the users of one specific operating system.

This will not be possible. The programming paradigms are different.

Linux versions are planned for Q2/2010.
Title: A multicore FLAC encoder
Post by: Lucho on 2009-11-16 15:12:39
Code: [Select]
FPSTREAM.EXE readfile in -f <wav-filename> + wavin wav -s in + fpflacenc enc -s wav [-<compression level>] + writefile out -s enc -f <flac-filename>



This is so confusing when it throw an error i dont know which one is argument 1 , argument 2 .... more simplified version will do the trick 

Apart from that good work.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-16 16:33:04
This is so confusing when it throw an error i dont know which one is argument 1 , argument 2 .... more simplified version will do the trick 

You can use a simple batch file:
Code: [Select]
@fpstream readfile in -f %1 + wavin wav -s in + fpflacenc enc -s wav + writefile out -s enc -f %2

If the name of the batch file is 'doflac.bat' then the cmdline is:
Code: [Select]
doflac <wav-filename> <flac-filename>

Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-16 17:25:59
OK, I've uploaded a new version. The link (http://www.thinkmeta.de/download/fpflac_beta.7z) is the same as before.

In order to use the SSE versions of 'fpFLAC' you have to copy them from either the 'sse2' or the 'sse4' folder into the main folder.

Edit: Sorry, I found a bug the SSE code (now I know why it was so fast...) . I had to rebuild the project and upload a new version.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-16 19:51:46
It would be interesting to compare it to CUDA FLAC.

Well, the "problem" is that FLAC encoding is already I/O-bound when running on quad-cores - at least on my system. So a CUDA FLAC or a >4-core-system will not run faster than my app.
Title: A multicore FLAC encoder
Post by: flacflac on 2009-11-17 17:58:46
Hi GeorgeFP,

thank you for this exciting multicore implementation.

I have a couple of problems using it, perhaps you can help out a bit:

1) Using Win7 x64, most compression settings seem to crash the tool: only if I keep don't specify it or use -8 does it work, and there seems to be a difference in filesize between the two, so I am not sure what the default compression setting is - can you reproduce this problem?  (it starts encoding but just doesn't finish)

2) The times I need to encode quickly are either during CD extraction using EAC or when I re-encode existing FLACs as I used lower compression setting before but would like to "upgrade".  Can you please tell me whether you are planning to support TRANSCODING existing FLACs?  Also, is the compression setting the only original FLAC argument the tool supports?

3) Is it possible to align commands with the original FLAC encoder to be able to simply substitute the current FLAC.exe with your development in EAC?

4) I add padding to my FLACs as I embed album art - unfortunately the padding command as specified in the FLAC documentation seems unsupported (http://flac.sourceforge.net/documentation_tools_flac.html (http://flac.sourceforge.net/documentation_tools_flac.html)) - implementation should not be a big problem, or am I mistaken? 

5) Is tag support a possibility in the near future?

Again, thank you!
ff
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-17 21:21:51
Thanks for your feedback!

Quote
1) Using Win7 x64, most compression settings seem to crash the tool: only if I keep don't specify it or use -8 does it work, and there seems to be a difference in filesize between the two, so I am not sure what the default compression setting is - can you reproduce this problem?  (it starts encoding but just doesn't finish)

I could reproduce the problem with "-1" and "-4". Both modes use loose mid-stereo which requires special handling. I thought, I made it right, but obviously not. It will be resolved in the next release.
BTW, the default is "-5".

Quote
2) The times I need to encode quickly are either during CD extraction using EAC or when I re-encode existing FLACs as I used lower compression setting before but would like to "upgrade".  Can you please tell me whether you are planning to support TRANSCODING existing FLACs?  Also, is the compression setting the only original FLAC argument the tool supports?

Transcoding is already possible:
Code: [Select]
FPSTREAM readfile in -f <old-flac-filename> + flacdec dec -s in + fpflacenc enc -s dec + writefile out -s enc -f <new-flac-filename>


Another option that my tool supports is "--lax" but I don't know what it does. It was easy to port it from the original FLAC code, so I did it.

Quote
3) Is it possible to align commands with the original FLAC encoder to be able to simply substitute the current FLAC.exe with your development in EAC?

Yes. I also plan to provide something like a "fpFLAC.exe" with the same command-line options as the original FLAC - for simple encodings.

Quote
4) I add padding to my FLACs as I embed album art - unfortunately the padding command as specified in the FLAC documentation seems unsupported (http://flac.sourceforge.net/documentation_tools_flac.html (http://flac.sourceforge.net/documentation_tools_flac.html)) - implementation should not be a big problem, or am I mistaken? 

This is not a problem, the code is already written, only the cmd-line option is missing. Which tool are you using to embed the album art? I'll test it with that.

Quote
5) Is tag support a possibility in the near future?

Yes. In the next release you should be able to set title, artist, album, year, track and genre. Just tell me if you need more.

Thanks again for your feedback!
Title: A multicore FLAC encoder
Post by: flacflac on 2009-11-18 02:10:34
Hi George, thank you for that detailed reply!

Quote
I could reproduce the problem with "-1" and "-4". Both modes use loose mid-stereo which requires special handling. I thought, I made it right, but obviously not. It will be resolved in the next release.
BTW, the default is "-5".


Good to know, I am using 8 anyways, but just thought I'd point that out.

Quote
Transcoding is already possible:
Code: [Select]
FPSTREAM readfile in -f <old-flac-filename> + flacdec dec -s in + fpflacenc enc -s dec + writefile out -s enc -f <new-flac-filename>


Oh, excellent!!!  Will test it out tomorrow!

Quote
Another option that my tool supports is "--lax" but I don't know what it does. It was easy to port it from the original FLAC code, so I did it.

This is an option that allows you to generate a rather non-conform FLAC file, which is not really advisable - but hey, any command that you can port is fine with me, just don't encourage people to use this one as they might end up with FLACs that don't play anymore through their streaming hardware. 

Quote
Yes. I also plan to provide something like a "fpFLAC.exe" with the same command-line options as the original FLAC - for simple encodings.

Great, I think it could make your development widely used, as you could tie it into foobar, eac and anything else without changing every tool around the current command-sequence.

Regarding padding:
Quote
This is not a problem, the code is already written, only the cmd-line option is missing. Which tool are you using to embed the album art? I'll test it with that.

Excellent!  I actually use www.mp3tag.de (http://www.mp3tag.de) to embed, and I just keep a small padding of 64kbyte (= -p 65536).  This way no rewrites become necessary when embedding.

Quote
Yes. In the next release you should be able to set title, artist, album, year, track and genre. Just tell me if you need more.


Awesome!  Perhaps you could implement this support in an "open" fashion, i.e. the way flac.exe does it:

Quote
-T FIELD=VALUE
Add a FLAC tag. The comment must adhere to the Vorbis comment spec (which FLAC tags implement), i.e. the FIELD must contain only legal characters, terminated by an 'equals' sign. Make sure to quote the comment if necessary. This option may appear more than once to add several comments. NOTE: all tags will be added to all encoded files.


So, as long as the Vorbis comment spec tag is used, it would work with any of these:

http://age.hobba.nl/audio/mirroredpages/ogg-tagging.html (http://age.hobba.nl/audio/mirroredpages/ogg-tagging.html)

I am not sure how, but EAC allows for storing of non-standard tags - I personally use the DISCID that comes from CDDB when encoding CD rips made with EAC (-T "GENRE=%m").  If you need some beta testing, let me know.

Again thank you, this could really rock!
ff
Title: A multicore FLAC encoder
Post by: Cokemonkey11 on 2009-11-18 02:58:12
Yes. In the next release you should be able to set title, artist, album, year, track and genre. Just tell me if you need more.


Is album art possible to be defined as a tag?
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-18 05:59:50
Is album art possible to be defined as a tag?

The option is "--picture=SPECIFICATION" in the original FLAC. I will add it.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-18 06:05:06
Awesome!  Perhaps you could implement this support in an "open" fashion, i.e. the way flac.exe does it:

Quote
-T FIELD=VALUE
Add a FLAC tag. The comment must adhere to the Vorbis comment spec (which FLAC tags implement), i.e. the FIELD must contain only legal characters, terminated by an 'equals' sign. Make sure to quote the comment if necessary. This option may appear more than once to add several comments. NOTE: all tags will be added to all encoded files.


Ok 
Title: A multicore FLAC encoder
Post by: Cokemonkey11 on 2009-11-18 06:09:38
Is album art possible to be defined as a tag?

The option is "--picture=SPECIFICATION" in the original FLAC. I will add it.


I meant in relation to my quoted post. You mentioned arist, album, title.. etc

Album Art should be one of those tags, unless I'm incorrect.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-18 12:25:00
Is album art possible to be defined as a tag?

The option is "--picture=SPECIFICATION" in the original FLAC. I will add it.


I meant in relation to my quoted post. You mentioned arist, album, title.. etc

Album Art should be one of those tags, unless I'm incorrect.

When I add the "-T" option it will be possible to write "-T ALBUMART=pic.png". AFAIK, only the filename will be saved then, not the image data.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-18 18:52:25
I have uploaded a new version.

The release notes are:
- Fixed problem with adaptive mid-side
- Added cmd-line option "-P <size>" (Adds PADDING block)
- Added cmd-line option "-T <field>=<value>" (Adds VORBIS comment)

I've tested this versions with all possible compression levels without problems.
I've also included a readme.txt that shows some features of the command-line

I've made a new benchmark test. This time I used compression level "-8". The results:

FLAC 1.2.1: 7 mins 44 secs
fpFLAC x64: 2 mins 13 secs

Speedup = 3.5x; Efficiency = 87.5%
Title: A multicore FLAC encoder
Post by: flacflac on 2009-11-19 08:17:26
I have uploaded a new version.

The release notes are:
- Fixed problem with adaptive mid-side
- Added cmd-line option "-P <size>" (Adds PADDING block)
- Added cmd-line option "-T <field>=<value>" (Adds VORBIS comment)

I've tested this versions with all possible compression levels without problems.
I've also included a readme.txt that shows some features of the command-line


Hi George, excellent update!!  Tagging (even random tags) is working, great job!  I have 3 issues:

1) Could you look into supporting the -V tag? (that's capital 'V' or --verify):

Quote
-V, --verify    
Verify the encoding process. With this option, flac will create a parallel decoder that decodes the output of the encoder and compares the result against the original. It will abort immediately with an error if a mismatch occurs. -V increases the total encoding time but is guaranteed to catch any unforseen bug in the encoding process.


V is very important to many people encoding from WAV as this option increases confidence that nothing got damaged during the encoding process - I don't know how complicated it is to implement that, probably not super easy.  It would be great, however!

2) I am having some trouble using the batch conversion mentioned in the readme for transcoding:

Code: [Select]
(from my 'doflacmulti.bat')
@fpstream readfile in*i -f "*fp" + flacdec dec*i -s in*i + fpflacenc enc*i -8 -P 65536 --verify -T DISCID=123 -s dec*i + writefile out*i -s enc*i -f "*n.flac"


I keep getting:

ERROR (enc*i): Syntax Error (argument: 16)
ERROR: Cannot create task 'fpflacenc'

I guess I must be missing something, perhaps you can point it out to me?

3) Would it be difficult to have existing tags preserved?  Especially for transcoding files this would be a very important feature.


But like I said, great update and I LOVE the speed of this - thank you so much.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-19 09:10:18
1) Could you look into supporting the -V tag? (that's capital 'V' or --verify):

It's not easy to implement this but there is a way. The biggest problem will be that you will not be able to specify just the -V tag. Instead you will have to add an additional FLAC decoder and a verifier task to the (very long) command line.

Quote
2) I am having some trouble using the batch conversion mentioned in the readme for transcoding:

Code: [Select]
(from my 'doflacmulti.bat')
@fpstream readfile in*i -f "*fp" + flacdec dec*i -s in*i + fpflacenc enc*i -8 -P 65536 --verify -T DISCID=123 -s dec*i + writefile out*i -s enc*i -f "*n.flac"


I keep getting:

ERROR (enc*i): Syntax Error (argument: 16)
ERROR: Cannot create task 'fpflacenc'

The first problem is the "--verify" option which is not supported.
The second problem is that you haven't specified a file mask (see (7) in readme.txt). The parser will treat "*fp" as a file name.

Quote
3) Would it be difficult to have existing tags preserved?  Especially for transcoding files this would be a very important feature.

I already have a plan for that 

Right now, I'm preparing the web site for the encoder with some explanations about how the algorithm works.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-11-23 09:34:11
Here is a screenshot how to configure EAC to use fpFLAC:

(http://www.thinkmeta.de/pix/eac.png)

Of course, you can set a compression level other than "-8".

On "EAC -> EAC options -> Tools", set the number of "simultaneous external compressor thread(s)" to 1.
Title: A multicore FLAC encoder
Post by: howesz on 2009-12-02 02:14:06
For ease of integration with EAC (i.e. so you can copy the text and paste it  ) here is the string I used...

readfile in -f %s + wavin wav -s in + tag tag "ARTIST=%a" "TITLE=%t" "ALBUM=%g" "YEAR=%y" "TRACK=%n" "GENRE=%m" + fpflacenc enc -s wav -8 -tag tag + writefile out -s enc -f %d


GeorgeFP,
How hard will it be to add replay gain support? (Please?)  With the addition of Verify support (mentioned above), I'll be all set 

My usual EAC FLAC 1.2.1b string is:

-8 --replay-gain -V -T "artist=%a" -T "title=%t" -T "album=%g" -T "date=%y" -T "tracknumber=%n" -T "genre=%m" %s

Thanks for the great program!!!  Keep up the great work!!!
Title: A multicore FLAC encoder
Post by: Wombat on 2009-12-02 02:36:31
You may use it together with metaflac with linking to cmd.exe as encoder in the way it is explained here in the Wiki for AAC. I am using this logic with Flacuda with no problem. You even can add the tags with Metaflac.
One downside of cause of these direct cmd-lines is that no album-gain can be set corretly.

http://wiki.hydrogenaudio.org/index.php?title=EAC_and_AAC (http://wiki.hydrogenaudio.org/index.php?title=EAC_and_AAC)

maybe this helps:
/c ""C:\path\CUETools.FlaCuda.exe" -8 --verify %s -o %d && "C:\path\metaflac.exe" --set-tag="Album=%g" --set-tag="Artist=%a" --set-tag="Title=%t" --set-tag="Tracknumber=%n" --set-tag="Date=%y" --set-tag="Genre=%m" --set-tag="Comment=FlaCuda 0.6 -8" %d"

good luck
Title: A multicore FLAC encoder
Post by: Chinch on 2009-12-02 05:25:17
interesting project you have going here. i know that as mentioned before... gregory chudov was working on a similar project called "flaCUDA" using the Nvidia CUDA api's... i always find this stuff interesting to watch though. i have a couple of questions for you:

1) the encoding algorithm is standardized, correct? that is, obviously it is going to provide lossless encoding/decoding, and compress to (at least) the same filesize as one encoded with the original FLAC.exe? i know there were some projects going on trying to increase performance/compression on this codec.... which i don't like, standards are made for reasons 

2) you say the current algorithm for encoding is i/o bound, i'm assuming this means the encoder can encode faster than the hard drive can read/supply the data, is this what you mean? unfortunately, this probably means that on faster systems, the speed improvement (vs. the original encoder) will have a tendency to decline the faster the system gets. fortunately, this also will improve with current and future tech like SSD drives... 

3) i would also like the verification function. as far as i know, isn't this a simple matter of computing the MD5 checksum of the raw audio data, encoding, then decompressing and comparing the MD5 values? (the MD5 checksum itself is stored within the FLAC file, obviously you know this) splitting up the threading and stuff could make this complicated if it was calculated in real-time, but if the MD5 was made based on the source WAV, then turn right back around and decode to a temp file, and compare. this would obviously add time to the overall process, plus require 'decoder' code implemented. the original FLAC program seems to do the "Verify OK" instantaneously (or maybe my computer is just really fast)... can you extrapolate on what would be necessary to include this function?  you already answered this in the above post 

4) i would be willing to write or help out with a GUI portion of this, or at least a frontend would be pretty easy to whip up to help simplify for some...


but the batch file posted, wouldn't this simple trick work -> copy the batch file to your desktop or Send To menu, (taking in %1 or argument as a variable) and then encoding that --- that is, basically drag n' drop the WAV file onto the shortcut icon (or send to menu) to encode the file?

Quote
One downside of cause of these direct cmd-lines is that no album-gain can be set correctly.
unless it was an image (single wav) rip, right?


i'm definitely going to test it out though. good work. i will report back my results.


noticed one comsmetic bug  in your command line help, when citing the examples of input/output, you still have reference to *MP3* files... i'm assuming just a copy and paste where you forgot to change? just letting you know 

Quote
Examples:

Encode one WAV file to one MP3:
FPSTREAM readfile in -f audio.wav
      + wavin wav -s in
      + fpmp3enc mp3enc -s wav --vbr-new
      + writefile out -s mp3enc -f audio.mp3

Encode one WAV file to one MP3 and one OGG with tags:
FPSTREAM readfile in -f audio.wav
      + wavin wav -s in
      + tag tag "TITLE=Audio Title" "ARTIST=Audio Artist"
      + fpmp3enc mp3enc -s wav --vbr-new -tag tag
      + vorbisenc oggenc -s wav -tag tag
      + writefile mp3file -s mp3enc -f audio.mp3
      + writefile oggfile -s oggenc -f audio.ogg
Title: A multicore FLAC encoder
Post by: Chinch on 2009-12-02 06:23:58
Alright! I'm impressed. I have to say that I didn't think the  improvement would be that great on my system, but here are my results (based on about a 53 minute CD):

original FLAC encoder (-8) = 1 minute, 15 seconds.   
and George FP's encoder (-8) = 24 seconds!   

i didn't believe it, so I did a bit-compare on the two files:

Code: [Select]
All tracks decoded fine, no differences found.
    
    Comparing:
    "X:\encode1.flac"
    "X:\encode2.flac"
    No differences in decoded data found.


and then I did a verify on the FLAC files, to make sure they were identical (audio data-wise):

Code: [Select]
Item: "X:\encode1.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.
    
    
    Item: "X:\encode2.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.



finally,  to prove that it wasn't EXACTLY THE SAME THING, i have noted the file  sizes and hashes of both FLAC files... different! both files are unique.

Code: [Select]
File: encode1.flac
    Size: 357MB (374,398,783 bytes)
    CRC-32: fa984f9e
       MD5: 6eb400219c22f1aa1525d13f5f7f1135
     SHA-1: 70aa7e4a65ed635eb78b52ef3d2f1b1b8b435692
    
    File: encode2.flac
    Size: 356MB (374,328,806 bytes)
    CRC-32: eac29087
       MD5: a8d830f2defbe50af368ce60a5a69460
     SHA-1: c356080e0e508d26b8d7b625229af0d2ca2dde7b


so George... you have made a believer of me! empirical evidence, right there for me and everyone else to see.... congrats!
I am about to do a second test on a second system, then I will post those results below.

Results from test #2:

original FLAC encoder (-8 compression) = 1 minute, 07 seconds
and George FP's encoder (-8 compression) = 21 seconds

just for kicks I tried flaCUDA...

flaCUDA encoder (-11 compression) = 58 seconds
flaCUDA encoder (-8 compression) = 16 seconds

Code: [Select]
E:\Temp>CUETools.FlaCuda.exe -11 encode.wav -o encode3.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
    Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 53.89x; 373021162 bytes in 00:00:58.8901034 seconds;
      
      E:\Temp>CUETools.FlaCuda.exe -8 encode.wav -o encode4.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
      Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 198.27x; 373969399 bytes in 00:00:16.0056281 seconds;


but there is a discrepancy.... flaCUDA's compression scheme goes up to 11, and that is what the 58 second result was based on... when I use compression -8, the encoding time ends up only taking 16 seconds. jeez. I'm not sure how the compression numbers crossover though. all 4 output files are identical to the original and each other (same audio MD5's), the two files produced by FLAC and fpFLAC both have a bitrate of 944, whereas the flaCUDA -11 encode shows a 940 bitrate, and the flaCUDA -8 encode shows a 943 bitrate (in foobar2000) -- all end-product FLAC files are sized between 355MB and 357MB, so negligible difference, size wise. *shrug* i have no idea.... already put waaaay too much thought into this as it is haha

anyways, with differing compression schemes, it's hard to make a solid conclusion.... other than that both of these two are VERY FAST encoders!

oh yeah, and of course, I forgot to include what specs I was using to test:

Intel i7-860 (2.8Ghz stock, 3.47Ghz in turbo mode -- it's a Quad-core with Hyper-Threading, so it sees it as 8 CPU's)
4GB DDR3-1600 RAM
Windows7 x64... I used of course the 64 bit version and the SSE4 instructions as George said in the readme
(and video card is Nvidia GeForce 9600 GSO)

Title: A multicore FLAC encoder
Post by: jamesbaud on 2009-12-02 10:00:14
Here is a screenshot how to configure EAC to use fpFLAC:

(http://www.thinkmeta.de/pix/eac.png)

Of course, you can set a compression level other than "-8".

On "EAC -> EAC options -> Tools", set the number of "simultaneous external compressor thread(s)" to 1.


Could you provide a screenshot to configure foobar to use fpFLAC?
Title: A multicore FLAC encoder
Post by: JacksonGrey on 2009-12-02 12:40:41
Could you provide a screenshot to configure foobar to use fpFLAC?


I'd like to second this request.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-03 08:50:42
Could you provide a screenshot to configure foobar to use fpFLAC?

Today, I've finished the implementation of dedicated tools for specific tasks. The names of the tools are self-explanatory: fpMP3, fpFLAC, fpFLAC2FLAC and fpFLAC2MP3. I will release them tomorrow.

Tomorrow, I will also provide new screenshots for EAC and foobar.
Title: A multicore FLAC encoder
Post by: gottkaiser on 2009-12-03 11:26:37
Wow, I'm impressed.
Thanks for all the work effort you put into this project!
Title: A multicore FLAC encoder
Post by: Glenda on 2009-12-03 13:14:38
Such a nice upgrade to EAC,  thankyou
Title: A multicore FLAC encoder
Post by: Glenda on 2009-12-03 13:22:34
Such a nice upgrade to EAC,  thankyou



Except I'm getting an error with some tracks
Title: A multicore FLAC encoder
Post by: Glenda on 2009-12-03 13:36:51
Such a nice upgrade to EAC,  thankyou



Except I'm getting an error with some tracks




No errors when running EAC under Win7x64 as admin.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 11:27:12
Hi!

The new version can be downloaded here:

64-Bit version (http://www.thinkmeta.de/download/fpStream64_1_0_0_19.7z)
32-Bit version (http://www.thinkmeta.de/download/fpStream32_1_0_0_19.7z) (not supported)

Just extract the archive and execute "setup.exe".

To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

Here is the screenshot for EAC:

(http://www.thinkmeta.de/pix/eac2.png)

In this example, I've used compression level 8.

If you like to add some tags, use the "-T" option, e.g.
Code: [Select]
-T "ARTIST=%a" -T "TITLE=%t" ...

EAC FAQ: Which flags can I use in the external compression scheme “User Defined MP3 Encoder”? (http://www.exactaudiocopy.de/en/index.php/support/faq)

I will upload the foobar screenshot in a few minutes.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 11:47:08
And here is the screenshot for foobar2000:
(http://www.thinkmeta.de/pix/foobar2000.png)

You should also set the number of converter threads to 1:

File -> Preferences -> Advanced -> Tools -> Converter -> Thread count : 1

Note: Because of a programming bug the encoder may hang in some (rare) cases. It's under investigation.
Title: A multicore FLAC encoder
Post by: gottkaiser on 2009-12-04 12:08:12
To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

I would like to use fpFLAC2MP3. But I'm not able to build the project. I'm just not into this subject.
Maybe somebody smart who owns all the needed software likes to build the project and upload it here in the forum? 

That would be very appreciated.
Title: A multicore FLAC encoder
Post by: johnsonlam on 2009-12-04 12:25:22
64-Bit version (http://www.thinkmeta.de/download/fpStream64_1_0_0_19.7z)
32-Bit version (http://www.thinkmeta.de/download/fpStream32_1_0_0_19.7z) (not supported)
Just extract the archive and execute "setup.exe".


Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 13:03:13
64-Bit version (http://www.thinkmeta.de/download/fpStream64_1_0_0_19.7z)
32-Bit version (http://www.thinkmeta.de/download/fpStream32_1_0_0_19.7z) (not supported)
Just extract the archive and execute "setup.exe".


Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!

I will try to change the setup language to English. Can you send me a screenshot of the failed installation? Then, I'll be able to do two things at once.
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 17:12:36
Considering that EAC only passes one file at a time and has its own internal ability to make use of multiple processes, can someone please explain how you derive a benefit from this program with EAC?
Title: A multicore FLAC encoder
Post by: pdq on 2009-12-04 18:51:54
So are you saying that it is pointless to use this version for ripping? That it is really only useful for batch converting?
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 18:57:57
I know I'm usually a sarcastic cynic, but this time I'm honestly asking for information.  After considering this some more, I'd like to see rationale for using it with foobar2000 as well.

To answer your question though, with my limited understanding of all this, yes it does seem quite pointless.
Title: A multicore FLAC encoder
Post by: Gregory S. Chudov on 2009-12-04 19:04:54
This encoder is useful, because it can use all the available processor cores when compressing a single file.
This is especially useful when ripping not as separate tracks, but as a single image.
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 19:10:53
So it's a departure from this?
http://www.hydrogenaudio.org/forums/index....showtopic=73790 (http://www.hydrogenaudio.org/forums/index.php?showtopic=73790)
Quote
The current version is still mostly single-threaded for single-file encoding but scales for multiple-files encoding.

Ok, seeing the initial post here I do see
Quote
The files were encoded one after the other, not in parallel.
...for the benchmark test.  Presumably demonstrating that the program does indeed process each file using more than one core.

So yeah, I would expect only a slight speedup for individual tracks unless there's a really big track that was ripped towards the end of the disc.
Title: A multicore FLAC encoder
Post by: pdq on 2009-12-04 19:20:03
Wouldn't the time to compress a single image file be a lot shorter than the time it took to rip it anyway, even using just a single core?
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 19:36:19
EAC doesn't start encoding with an external compressor until after the file is ripped.
Title: A multicore FLAC encoder
Post by: pdq on 2009-12-04 20:25:11
I'm just saying that the time to encode after you finish ripping is fairly short relative to the time spent ripping. It probably takes you longer to replace that CD with the next one.
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 20:29:47
On my system it still takes longer to encode @ -8 with flac 1.2.1 than it takes to rip unless the disc is giving the drive trouble.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 20:37:39
How hard will it be to add replay gain support? (Please?)  With the addition of Verify support (mentioned above), I'll be all set 

Oops, I missed this post.

Replay gain should not be a problem. I will add it in one of the next versions.
And verify support.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 21:02:21
1) the encoding algorithm is standardized, correct? that is, obviously it is going to provide lossless encoding/decoding, and compress to (at least) the same filesize as one encoded with the original FLAC.exe? i know there were some projects going on trying to increase performance/compression on this codec.... which i don't like, standards are made for reasons

The encoding algorithm is exactly the same (except some SSE optimizations) as the original FLAC.

Quote
2) you say the current algorithm for encoding is i/o bound, i'm assuming this means the encoder can encode faster than the hard drive can read/supply the data, is this what you mean? unfortunately, this probably means that on faster systems, the speed improvement (vs. the original encoder) will have a tendency to decline the faster the system gets. fortunately, this also will improve with current and future tech like SSD drives...

The encoder is embarassingly parallel so sooner or later the I/O will become the bottleneck.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 21:21:16
Considering that EAC only passes one file at a time and has its own internal ability to make use of multiple processes, can someone please explain how you derive a benefit from this program with EAC?

If it's possible to utilize all cores with one single file, why would you want to process more than one file in parallel?
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 21:29:50
Perhaps you could have made it a little more clear as to what your encoder does so that idiots like me wouldn't have to ask such stupid questions.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-04 21:43:38
Perhaps you could have made it a little more clear as to what your encoder does so that idiots like me wouldn't have to ask such stupid questions.

So, you did NOT visit my web site? Take a look here (http://www.thinkmeta.de/en/fiberpool_case_studies_fpflac_algorithm.html).

BTW, it seems I've found the bug I've mentioned before - one hour full-speed encoding without a hang.
The new version is online, the names are the same.
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-04 21:50:46
Well no.  Why didn't you provide that link in the first place?
Title: A multicore FLAC encoder
Post by: jamesbaud on 2009-12-05 13:32:31
To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

I would like to use fpFLAC2MP3. But I'm not able to build the project. I'm just not into this subject.
Maybe somebody smart who owns all the needed software likes to build the project and upload it here in the forum? 

That would be very appreciated.


Any chance the built files could be hosted on Rarewares?
The "old" fpMP3Enc is already there.
http://www.rarewares.org/mp3-others.php (http://www.rarewares.org/mp3-others.php)
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-07 18:55:56
A new version with two bug fixes is available for download:

64-Bit version (http://www.thinkmeta.de/download/fpStream64_1_0_0_20.7z)
32-Bit version (http://www.thinkmeta.de/download/fpStream32_1_0_0_20.7z) (not supported)

Bug#1: Application hung (became idle) sometimes
Bug#2: Memory access violation

Tomorrow I will start work on the -V option and the replay gain feature.
Title: A multicore FLAC encoder
Post by: JacksonGrey on 2009-12-07 23:59:01
I keep getting an error that a number of files 'failed to register'. Anyone else experiencing this? Files are installed into the Program Files directory, but any conversion fails in Foobar.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-08 06:14:34
I keep getting an error that a number of files 'failed to register'. Anyone else experiencing this? Files are installed into the Program Files directory, but any conversion fails in Foobar.

The Visual C++ runtimes are not included in the package. You can download them here:

32-Bit (http://www.microsoft.com/downloads/details.aspx?familyid=A5C84275-3B97-4AB7-A40D-3802B2AF5FC2&displaylang=en)
64-Bit (http://www.microsoft.com/downloads/details.aspx?familyid=BA9257CA-337F-4B40-8C14-157CFDFFEE4E&displaylang=en)

You can also try to register the DLLs manually:
1. Open a command prompt as administrator
2. Go to the installation directory
3. Type
Code: [Select]
regsvr32 fpFLAC.dll
regsvr32 fpFLAC_SSE2.dll
regsvr32 fpFLAC_SSE4.dll
regsvr32 fpMP3.dll
regsvr32 fpMP3_SSE2.dll
regsvr32 fpMP3_SSE4.dll
regsvr32 fpStreamLib.dll
regsvr32 fpWave.dll
regsvr32 lFLAC.dll
regsvr32 lVorbis.dll

Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-08 12:12:46
I've found the problem with the installer: Several files were missing in the packages so I had to rebuild them again. Sorry for that. The above links still work.

You may get an error when registering fpFLAC_SSE2.dll or fpFLAC_SSE4.dll. This happens when your system does not support SSE2 or SSE4. Just press "Continue" then.

The new installer has updated versions of fpFLAC and fpFLAC2FLAC where you can pass the -V (verify) option.
Title: A multicore FLAC encoder
Post by: greynol on 2009-12-08 16:34:08
I'd love to see this done with metaflac so that I can do quick replaygain scanning.
Title: A multicore FLAC encoder
Post by: NuSkooler on 2009-12-23 17:16:47
I cannot wait for this to be ready for production so to speak!

Any chance @ GPU support?
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-24 11:47:03
I cannot wait for this to be ready for production so to speak!

Well, the truth is, I don't know if fpFLAC has a future. The feedback is very, very poor.

Quote
Any chance @ GPU support?

I will add GPU support into my framework and if the fpFLAC project will be continued it will also support GPUs.
Title: A multicore FLAC encoder
Post by: Bylie on 2009-12-24 13:59:21
Well, the truth is, I don't know if fpFLAC has a future. The feedback is very, very poor.

Well to be honest I've been following all of the new projects with a lot of interest and am really impressed by the progress made. Who wouldn't be:
But, personally, I'm not very inclined to switch to a new, unproven piece of software even if it incorporates proven code and/or algorithms like FLAC, WavPack, LAME, ...

Maybe a better approach to this would be, to somehow try, to integrate these new projects into the existing, proven, codebase supervised and/or publicly supported by the original developers trying to maintain the same stability and quality of the original software. This might even be a different branch at first but I really think the "umbrella effect" could have a positive effect on the acceptance of the new code/projects by the users which ultimately will result in more useage and feedback.

I know this is sort of an idealistic vision because in reality it might just not be that easy due to a number of realworld problems/conflicts (licensing, time, ...) but I really think it could give these new projects a lot more visibility and it might even give the original software a new momentum.
Title: A multicore FLAC encoder
Post by: Gregory S. Chudov on 2009-12-24 14:20:25
Agree. Josh, if you are interested, i'm ready to spend some time helping to incorporate CUDA code.
Title: A multicore FLAC encoder
Post by: skamp on 2009-12-24 15:30:15
Josh, if you are interested, i'm ready to spend some time helping to incorporate CUDA code.

GPU encoding in flac through libcuda under linux would be ideal (awesome).
Title: A multicore FLAC encoder
Post by: _m²_ on 2009-12-24 23:56:27
I cannot wait for this to be ready for production so to speak!

Well, the truth is, I don't know if fpFLAC has a future. The feedback is very, very poor.

Well, there's some feedback I can give you. I installed it and deleted shortly after. I'm not an user, but I can tell you why.

1. For me FLAC is dead, it's too weak, so you came too late. Multithreaded wavpack or mac would probably make more sense.

2. The last command line that I used was
flac -8 -A tukey(0.5) -A flattop -r 0,8 -V
and I considered adding -p, but it was a bit too slow for the gain. Overall it's slightly stronger than -8.

If fpFLAC allowed me to compress slightly better with reasonable speed (and I wouldn't be shifting to TAK), I would use it, though probably with verification done by regular FLAC during the testing period.
However, fpFLAC doesn't accept these switches, so (I guess) it's even weaker than regular FLAC. For me - pointless.

3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.

4. No Uninstaller. That's extremely annoying. I guessed that I should use the installer, but if I didn't keep it (for these several minutes  ), I would just delete the program and have rubbish left in the registry and possibly other places too. And I wouldn't assume that everybody will guess, even considering that the target audience is rather advanced.

5. FLAC is (pendrive) portable, fpFLAC - I don't know. For me, it's very important. The only non-portable program that I found worth using is Visual Studio.
Title: A multicore FLAC encoder
Post by: Qest on 2009-12-25 00:02:37
It would be outstanding if the defacto flac tools had support for multiple cores (like fpFLAC) and gpu's (like FLACuda)! That would be the best of all worlds.
Title: A multicore FLAC encoder
Post by: NuSkooler on 2009-12-25 00:03:22
1. For me FLAC is dead, it's too weak, so you came too late. Multithreaded wavpack or mac would probably make more sense.


I doubt this is any reason for this particular project to receive minimal attention from outsiders. FLAC is the defacto standard for lossless audio. Sure, WavPack & others have gained some ground (especially around here) but in the "real world" you're going to find FLAC. I think some of the reason is people simply do not know of this. I use FLAC on a every day basis and just found out about this a few days ago. Also, until it can replace flac.exe with confidence, obviously it won't be used by many.

Quote
3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.


I agree. Especially for "utility" type stuff. My suggestion: Statically link everything needed into a fpFLAC.exe. I believe this will make it much more open to the masses.
Title: A multicore FLAC encoder
Post by: Zarggg on 2009-12-25 04:02:55
Will this program yield any benefit to a (single core) Pentium 4 with Hyperthreading enabled? My immediate thought would be "no", but I still want to ask.

I apologize if this seems like a dumb question, but it's not very obvious to me and I'm only just starting to learn about how multi-core processors work.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-25 12:33:44
But, personally, I'm not very inclined to switch to a new, unproven piece of software even if it incorporates proven code and/or algorithms like FLAC, WavPack, LAME, ...

Maybe a better approach to this would be, to somehow try, to integrate these new projects into the existing, proven, codebase supervised and/or publicly supported by the original developers trying to maintain the same stability and quality of the original software. This might even be a different branch at first but I really think the "umbrella effect" could have a positive effect on the acceptance of the new code/projects by the users which ultimately will result in more useage and feedback.

Believe me, if I had the chance to integrate my framework into the existing C code I would have done so. But multi-core programming is very different to serial programming and thus, I had to write fpFLAC from scratch.

Of course, you can wait until the original developers will add multi-core support to their projects. I don't know of any activities going in this direction, neither for FLAC nor for LAME.
Title: A multicore FLAC encoder
Post by: Bylie on 2009-12-25 17:43:38
Believe me, if I had the chance to integrate my framework into the existing C code I would have done so. But multi-core programming is very different to serial programming and thus, I had to write fpFLAC from scratch.

Of course, you can wait until the original developers will add multi-core support to their projects. I don't know of any activities going in this direction, neither for FLAC nor for LAME.

I really try to not underestimate the amount of work and research that goes into these projects and I can surely understand that saying something like "integration will yield benefit x and y" might sound much easier than actually doing it!
However, the problem will remain for projects like fpFLAC and others that they are seen more like a proof of concept and hence are more interesting for developers and testers which won't make a large userbase. This might actually not be a problem if the goal is to provide a proof of concept, but if the idea is to get more and more realworld usage/feedback I think somewhere along the line the proof of concept stage has to evolve into something more users will (want to) use. This is where the "umbrella effect" or "public developer support" might play a role as it gives visibility to these projects beyond this forum and also, kind of, assures the large group of endusers that it's OK to also use this for anything else than testing.
Title: A multicore FLAC encoder
Post by: Fandango on 2009-12-25 18:29:29
Uhm, I would happily replace all single-threaded command line encoders right away, with multi-core + GPU utilizing encoders. It annoys me everytime I transcode a single large file, that more than half of my CPU is idle, while I am wating... waiting... waiting...
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-25 19:51:08
Quote
3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.


I agree. Especially for "utility" type stuff. My suggestion: Statically link everything needed into a fpFLAC.exe. I believe this will make it much more open to the masses.

My "streaming framework" which is similar to DirectShow uses the registry to find the components/filters. I think this is a better approach than putting everything into a single file, especially when you have optimizations for different processor features (SSE, CUDA, Stream, ...)
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-25 21:11:57
I really try to not underestimate the amount of work and research that goes into these projects and I can surely understand that saying something like "integration will yield benefit x and y" might sound much easier than actually doing it!
However, the problem will remain for projects like fpFLAC and others that they are seen more like a proof of concept and hence are more interesting for developers and testers which won't make a large userbase. This might actually not be a problem if the goal is to provide a proof of concept, but if the idea is to get more and more realworld usage/feedback I think somewhere along the line the proof of concept stage has to evolve into something more users will (want to) use. This is where the "umbrella effect" or "public developer support" might play a role as it gives visibility to these projects beyond this forum and also, kind of, assures the large group of endusers that it's OK to also use this for anything else than testing.

I agree with you. For me, it's ok if fpFLAC is seen as a proof of concept and the project can end here right away.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-25 21:20:43
Will this program yield any benefit to a (single core) Pentium 4 with Hyperthreading enabled?

I don't know. I would say "no" or "not much".
Title: A multicore FLAC encoder
Post by: skamp on 2009-12-25 21:44:31
Encoding two files at once on my Intel Atom with hyperthreading typically yields a 40-50% improvement, which isn't negligible.
Title: A multicore FLAC encoder
Post by: Fandango on 2009-12-25 22:05:44
Will this program yield any benefit to a (single core) Pentium 4 with Hyperthreading enabled?



I don't know. I would say "no" or "not much".




Having previously owned a HT P4, I'd say it will.

Title: A multicore FLAC encoder
Post by: NuSkooler on 2009-12-26 02:24:18
My "streaming framework" which is similar to DirectShow uses the registry to find the components/filters. I think this is a better approach than putting everything into a single file, especially when you have optimizations for different processor features (SSE, CUDA, Stream, ...)


The reasons I suggest statically linking everything into a monster executable is portability. flac.exe for example can be dropped on a box and used right away. People like that.

Also, you can still optimize for various processor features in a single executable (e.g. dynamically use funcXXX() vs funcYYY()). Maybe I'm missing what you're trying to convey here?
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-26 08:27:29
The reasons I suggest statically linking everything into a monster executable is portability. flac.exe for example can be dropped on a box and used right away. People like that.

I understand that. Static linking would work for fpFLAC.

One of the next features will be AVI processing which I need for my next project 'fp264'. Because you cannot know how video and audio were encoded in a file you cannot link all codecs statically. So, another solution is needed to check if the system provides the components that are necessary to decode a stream. I've chosen the registry for that.

Another benefit of component based design will be, that it will be possible to write applications/frontends in any language and not only in C++.
Title: A multicore FLAC encoder
Post by: NuSkooler on 2009-12-26 16:17:13
Another benefit of component based design will be, that it will be possible to write applications/frontends in any language and not only in C++.


Absolutely. Having everything component based framework and providing one "drop in" fpFLAC.exe binary is the best of both worlds.

As for the codecs, is there not a standard way they that they are already layed out in the Registry? If I install a codec, any application supporting codecs of said type can find them. I don't think there is much to invent there.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-26 19:05:45
Another benefit of component based design will be, that it will be possible to write applications/frontends in any language and not only in C++.


Absolutely. Having everything component based framework and providing one "drop in" fpFLAC.exe binary is the best of both worlds.

fpFLAC is file based which is not always the best solution. For example, if you want to stream lossless audio (with unknown length) over the network. In this case, it's better to have the codec as a component where input and output can be controlled programmatically - in any language.

Quote
As for the codecs, is there not a standard way they that they are already layed out in the Registry? If I install a codec, any application supporting codecs of said type can find them. I don't think there is much to invent there.

You are right but depending on the used framework (VfW, ACM, DirectShow, Windows Media) the codecs are registered in different paths. I do exactly the same.

The first step will be to support legacy VfW or ACM codecs but later I or a 3rd party will provide multi-core-aware codecs.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2009-12-26 19:18:34
BTW, I've decided to add some missing command-line options into fpFLAC (-b, -m, -M, -e, -A, -l, -p, -q, -r and --replay-gain).
Title: A multicore FLAC encoder
Post by: Meeko on 2009-12-28 12:52:41
So I'm guessing a multicore FLAC encoder would only be most useful going from WAV or lossless files on a harddrive to FLACs?  I've been reading this thread with interest, but I usually rip from CD to FLAC using foobar2000 and my machine easily can convert to FLAC faster than the drive can read the disc.  Would I benefit from this at all?
Title: A multicore FLAC encoder
Post by: Fandango on 2009-12-28 14:22:01
You are right, multi-threaded encoders are for faster transcoding of already ripped audio.
Title: A multicore FLAC encoder
Post by: clintb on 2010-01-29 15:06:26
Any chance someone could whip up a batch file for this bad boy to use fpFLAC2FLAC.exe, somewhat like Synthetic Soul's flac-113.bat? My batch skills are pure rubbish. I have a bunch of files at flac 1.1.x and want to take all to 1.2.1.

Thanks for any help offered!

Example attached:
Code: [Select]
@ECHO OFF
CLS

REM FLAC-113.BAT Written by Neil Popham, 2007 ([email=neilpopham@bigfoot.com]neilpopham@bigfoot.com[/email])

SET batchname=FLAC-113.BAT
SET version=2.2.5
TITLE=%batchname%, Version %version%

REM ######################################################
SET pathToFLAC="C:\Apps\Encoders\FLAC\FLAC.EXE"
SET flacOptions=-8
SET retainListOfFailedFiles=1
SET retainListOfProcessedFiles=0
SET retainListOfSuccessfulFiles=0
SET flacVersion=121
REM ######################################################

REM Check whether a valid file or folder has been passed
IF NOT EXIST %1 DO GOTO NoParams

REM Set counters
SET /A noProcessed=0
SET /A noErrors=0
SET /A noEncoded=0
SET /A noSkipped=0
SET /A srcBytes=0
SET /A dstBytes=0

REM Set path to the list of failed files depending on whether it is to be kept or not
IF [%retainListOfFailedFiles%] EQU [1] (
 SET pathForFails=%~dp0%
) ELSE (
 SET pathForFails=%TEMP%\
)

REM Set path to the list of successful files depending on whether it is to be kept or not
IF [%retainListOfSuccessfulFiles%] EQU [1] (
 SET pathForSuccess=%~dp0%
) ELSE (
 SET pathForSuccess=%TEMP%\
)

REM Set path to the list of files to process depending on whether it is to be kept or not
IF [%retainListOfProcessedFiles%] EQU [1] (
 SET pathForList=%~dp0%
) ELSE (
 SET pathForList=%TEMP%\
)

REM Clear file used to store failed filenames
IF EXIST "%pathForFails%flac-113-failed.txt" DEL "%pathForFails%flac-113-failed.txt"

REM Clear file used to store successful filenames
IF EXIST "%pathForSuccess%flac-113-successful.txt" DEL "%pathForSuccess%flac-113-successful.txt"

REM Clear file used to store the list of files to process
IF EXIST "%pathForList%flac-113-processed.txt" DEL "%pathForList%flac-113-processed.txt"

REM Check whether the parameter is a single file or a folder
IF %~z1 EQU 0 (CALL :FolderAction %1) ELSE (CALL :FileAction %1)

REM Calculated bytes saved
SET /A dffSize=%srcSize%-%dstSize%

REM Report results
ECHO ________________________________________________________________________
ECHO.
ECHO %noProcessed% file(s) processed
IF EXIST "%~dp0FLACGETV.EXE" ECHO %noSkipped% file(s) skipped (already latest version)
ECHO %noEncoded% file(s) encoded
ECHO %noErrors% file(s) returned an error
IF %noErrors% GTR 0 CALL :ReportFiles
ECHO.
ECHO Settings used: FLAC v.%flacVersion% '%flacOptions%'
ECHO Bytes Saved:  %dffSize% bytes

REM Pause
ECHO.
ECHO Press any key to exit
PAUSE >NUL

REM End
GOTO:EOF

REM ==========================================
REM  FileAction : single file has been passed
REM ==========================================
:FileAction
REM Do we have a FLAC file?
IF /I [%~x1] EQU [.flac] (
 CALL :Encode %1
) ELSE (
 REM Do we have a TXT file?
 IF /I [%~x1] EQU [.txt] (
  REM Process the list of files in the file
  SET pathForList="%~dp1"
  CALL :ProcessList "%~nx1"
 ) ELSE (
  GOTO NoParams
 )
)
GOTO:EOF

REM =======================================
REM  FolderAction : folder has been passed
REM =======================================
:FolderAction
REM Check folder and subfolders for FLAC files,call ListFiles for each file, and then Encode
ECHO Processing: "%~1"
ECHO.
REM Create temporary list of files to process
ECHO Creating list of files to process
ECHO.
FOR /R %1 %%G IN (*.flac) DO CALL :ListFiles "%%G"
CALL :ProcessList flac-113-processed.txt
GOTO:EOF

REM =============================================
REM  ListFiles : Create list of files to process
REM : %1 The FLAC file to add
REM =============================================
:ListFiles
ECHO %1>>"%pathForList%flac-113-processed.txt"
GOTO:EOF

REM ============================================
REM  ProcessList : Encode all files in the list
REM   : %1 The text list of files
REM ============================================
:ProcessList
REM Encode each file
PUSHD %pathForList%
FOR /F "tokens=* delims=" %%H IN (%~s1) DO CALL :Encode %%H
POPD
IF [%retainListOfFailedFiles%] EQU [1] (
 REM ECHO.>>"%pathForList%flac-113-processed.txt"
 REM ECHO List created at %DATE% %TIME%>>"%pathForList%flac-113-processed.txt"
) ELSE (
 DEL "%pathForList%flac-113-processed.txt"
)
GOTO:EOF


REM =====================================
REM  Encode : Encode FLAC file
REM : %1 The FLAC file to Encode
REM =====================================
:Encode
REM Check the vendor string to see if the file is already the latest version
IF EXIST "%~dp0FLACGETV.EXE" (
 ECHO Checking Vendor: "%~1"
 ECHO ________________________________________________________________________
 "%~dp0FLACGETV.EXE" %1
 ECHO.
)
IF %ERRORLEVEL% EQU %flacVersion% (
 SET /A noSkipped+=1
 ECHO File is already the latest version [%flacVersion%]
) ELSE (
 REM Test the original
 ECHO Testing: "%~1"
 ECHO ________________________________________________________________________
 %pathToFLAC% -t %1
 REM If an error code has been returned increase the error counter
 IF ERRORLEVEL 1 (
  SET /A noErrors+=1
  ECHO  %1 [T]>>"%pathForFails%flac-113-failed.txt"
  ECHO.
  ECHO File failed testing.
 ) ELSE (
  REM Encode the file
  ECHO.
  ECHO Encoding: "%~1"
  ECHO ________________________________________________________________________
  %pathToFLAC% %flacOptions% -V -o "%~dpn1-flac-113.flac" %1
  REM If an error code has been returned increase the error counter
  IF ERRORLEVEL 1 (
  SET /A noErrors+=1
  ECHO  %1 [E]>>"%pathForFails%flac-113-failed.txt"
  ECHO.
  ECHO File failed encoding.
  IF EXIST "%~dpn1-flac-113.flac" DEL "%~dpn1-flac-113.flac"
  ) ELSE (
  SET /A noEncoded+=1
  CALL :IncreaseDestinationBytes "%~dpn1-flac-113.flac"
  CALL :IncreaseSourceBytes %1
  ECHO %1>>"%pathForSuccess%flac-113-successful.txt"
  MOVE /Y "%~dpn1-flac-113.flac" %1
  )
 )
)
ECHO.
ECHO.
REM Increase file counter
SET /A noProcessed+=1
GOTO:EOF

REM ==========================================
REM  ReportFiles : List the files that failed
REM ==========================================
:ReportFiles
ECHO.
ECHO Files that failed:
ECHO.
TYPE "%pathForFails%flac-113-failed.txt"
REM Delete the temporary file if not required, or add footer if keeping
IF [%retainListOfFailedFiles%] EQU [1] (
 REM ECHO.>>"%pathForFails%flac-113-failed.txt"
 REM ECHO List created at %DATE% %TIME%>>"%pathForFails%flac-113-failed.txt"
) ELSE (
 DEL "%pathForFails%flac-113-failed.txt"
)
ECHO.
GOTO:EOF

REM =====================================================
REM  NoParams : neither a file or folder has been passed
REM =====================================================
:NoParams
ECHO To run this script, please pass a valid file or folder as a parameter.
ECHO.
ECHO Press any key to exit
PAUSE >NUL
GOTO:EOF

:IncreaseSourceBytes
SET /A srcSize+=%~z1
GOTO:EOF

:IncreaseDestinationBytes
SET /A dstSize+=%~z1
GOTO:EOF


REM VERSION HISTORY
REM ===============
REM Version  Date Description
REM ================================
REM 1.0.0 2007-01-07  Initial Version
REM 1.1.0 2007-01-07  Now encodes to temporary file and then moves over source
REM 1.1.1 2007-01-07  Minor formatting changes
REM 2.0.0 2007-01-07  Now works on XP (creates a temporary list of files to process)
REM 2.1.0 2007-01-09  Fixed bug with ERRORLEVEL. T(est)/E(ncode) marker added for failed files
REM 2.2.0 2007-01-09  Will skip 1.1.3 files if FlacGetV.exe is same directory as this file
REM 2.2.1 2007-01-10  Added retainListOfFailedFiles and retainListOfProcessedFiles flags
REM 2.2.2 2007-01-12  Added flacVersion
REM 2.2.3 2007-01-28  Added retainListOfSuccessfulFiles
REM 2.2.4 2007-01-28  Now accepts a text list of files as input
REM 2.2.5 2007-02-16  Fixed FlacGetV bug. Now shows bytes saved
Title: A multicore FLAC encoder
Post by: flacflac on 2010-02-06 11:11:54
Well, there's some feedback I can give you. I installed it and deleted shortly after. I'm not an user, but I can tell you why.

1. For me FLAC is dead, it's too weak, so you came too late. Multithreaded wavpack or mac would probably make more sense.
etc.

Wow, really? There is close to zero hardware support out there for either of these codecs, so how would other codecs make more sense than the one used by the majority here at HA?  Also, what's the rationale behind switching to TAK, a closed-source codec without hardware support? Good luck switching to the next codec that comes along... .

3. Installers are annoying. Regular 7zip package would be better. Can't you make it work w/out registering these dlls? Also, I got error during registration of smth with SSE4 in the name. XP 32bit, SP3, Core 2 Duo. Program worked at least long enough to show help, seeing lacking switches, I didn't even bother to compress something.

Man, you haven't even used it at all?  I agree that this program would benefit from not coming with an installer but just an executable you can drop anywhere - but this is a work in progress and imho should not be knocked for that. 

The only non-portable program that I found worth using is Visual Studio.

Haha, good one.

@George
I hope you continue your development.  I can see about 40% speed increase using the command line to convert from a single WAV to a FLAC, yet I seem to be losing this advantage when turning a large WAV into multiple smaller FLACs using a CUE sheet in Foobar. 

My Foobar command line:

Code: [Select]
-8 --threads 2 -P 65536 -V -f %s %d

Is it expected to not work well with multiple files? Is this my I/O limiting the speed?  I am running a Core2Duo@2GHz.

Also, one more thing: I noticed that when doing a command-line encode there is no mentioning of a "-V"-verification happening - is it going on in the background?  Would it be possible to make fpflac  a bit more verbose and tell me the result of this verification?

Thanks for this development.

ff
Title: A multicore FLAC encoder
Post by: Synthetic Soul on 2010-02-06 13:29:29
Any chance someone could whip up a batch file for this bad boy to use fpFLAC2FLAC.exe, somewhat like Synthetic Soul's flac-113.bat? My batch skills are pure rubbish. I have a bunch of files at flac 1.1.x and want to take all to 1.2.1.
That batch file was superceded by the one in this thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=61511).  Thee are some concerns about a few weird characters in filenames, IIRC.

For the record though, I would assume that you'd just need to change the following lives near the very top:

Code: [Select]
SET pathToFLAC="C:\Apps\Encoders\FLAC\FLAC.EXE"
SET flacOptions=-8

I haven't been following this thread, so don't know what to suggest as replacements I'm afraid.

That is, if this encoder will handle FLAC files as a source.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2010-02-06 18:04:43
Is it expected to not work well with multiple files? Is this my I/O limiting the speed?  I am running a Core2Duo@2GHz.

In foobar2000, you have to set the number of threads to 1:

File -> Preferences -> Advanced -> Tools -> Converter -> Thread count : 1

Quote
Also, one more thing: I noticed that when doing a command-line encode there is no mentioning of a "-V"-verification happening - is it going on in the background?  Would it be possible to make fpflac  a bit more verbose and tell me the result of this verification?

Try to download the latest version from MY site. I've implemented this option some time ago.
Title: A multicore FLAC encoder
Post by: howesz on 2010-07-10 02:24:25
BTW, I've decided to add some missing command-line options into fpFLAC (-b, -m, -M, -e, -A, -l, -p, -q, -r and --replay-gain).


Are these options included in fpflac.exe (dated Feb 22, 2010) from the 64-bit v1.0.0.21 installer download?  Because I get an error when I use this command line:

-f %s %d -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "YEAR=%y" -T "TRACK=%n" -T "GENRE=%m" -8 -V --replay-gain

And again, thanks for your time and efforts!!  Multi-core is here to stay...

Zach
Title: A multicore FLAC encoder
Post by: Glenda on 2010-08-07 09:36:06
BTW, I've decided to add some missing command-line options into fpFLAC (-b, -m, -M, -e, -A, -l, -p, -q, -r and --replay-gain).


Are these options included in fpflac.exe (dated Feb 22, 2010) from the 64-bit v1.0.0.21 installer download?  Because I get an error when I use this command line:

-f %s %d -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "YEAR=%y" -T "TRACK=%n" -T "GENRE=%m" -8 -V --replay-gain

And again, thanks for your time and efforts!!  Multi-core is here to stay...

Zach



Same here what command line should we use in EAC?
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2010-08-29 11:14:49
Same here what command line should we use in EAC?


Sorry for the late reply. I will post screenshots for EAC and foobar2000 configuration soon.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2010-09-03 11:11:37
BTW, I've decided to add some missing command-line options into fpFLAC (-b, -m, -M, -e, -A, -l, -p, -q, -r and --replay-gain).


Are these options included in fpflac.exe (dated Feb 22, 2010) from the 64-bit v1.0.0.21 installer download?  Because I get an error when I use this command line:

-f %s %d -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "YEAR=%y" -T "TRACK=%n" -T "GENRE=%m" -8 -V --replay-gain

And again, thanks for your time and efforts!!  Multi-core is here to stay...

Zach


The --replay-gain option is currently not implemented so if you remove that it should work.

BTW, a new version (x64 only) is available for download which doesn't need an installer anymore, and I'm working on a Linux port. So, the project is not dead, yet 

George
Title: A multicore FLAC encoder
Post by: gimp on 2010-09-16 22:09:53
BTW, I've decided to add some missing command-line options into fpFLAC (-b, -m, -M, -e, -A, -l, -p, -q, -r and --replay-gain).


Are these options included in fpflac.exe (dated Feb 22, 2010) from the 64-bit v1.0.0.21 installer download?  Because I get an error when I use this command line:

-f %s %d -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "YEAR=%y" -T "TRACK=%n" -T "GENRE=%m" -8 -V --replay-gain

And again, thanks for your time and efforts!!  Multi-core is here to stay...

Zach


The --replay-gain option is currently not implemented so if you remove that it should work.

BTW, a new version (x64 only) is available for download which doesn't need an installer anymore, and I'm working on a Linux port. So, the project is not dead, yet 

George


Removed --replay-gain but it still doesn't work for me, even after upgrading to fpStream_1_0_0_23.  Any suggestions?
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2010-09-17 07:16:10
Removed --replay-gain but it still doesn't work for me, even after upgrading to fpStream_1_0_0_23.  Any suggestions?


Thanks for the feedback. I've found a problem when using the -V option. I will upload a fixed version this weekend.

BTW, I've merged all optimized version to one which detects processor features automatically. And the code is ported to VS2010 but I haven't tested yet if it can be compiled with the Express edition.

George
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2010-09-19 09:05:43
An updated version is now available on my site.

I've tested fpFLAC with the latest EAC version using the above arguments without errors.

Note 1: During my tests fpFLAC crashed from time to time when it was started by EAC while it ran fine from the command line. It seems (!) to be fixed now, because it didn't crash again in the last test runs.

Note 2: Don't forget to set the numbers of threads to 1 in EAC (EAC -> EAC options -> Tools).

Note 3: The compression performance would be better if EAC would pass all arguments for all files in a single call. In a future version I will add the option to read the arguments from a file.
Title: A multicore FLAC encoder
Post by: Glenda on 2011-01-17 07:47:23
The encoder flags errors on every attempt with EAC,  possibly not compatible with the latest version of eac
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2011-01-17 17:48:22
The encoder flags errors on every attempt with EAC,  possibly not compatible with the latest version of eac


I've downloaded "eac-1.0beta1.exe" and configured the fpFLAC arguments with -f %s %d -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "YEAR=%y" -T "TRACK=%n" -T "GENRE=%m" -8 -V. I've also added spaces in the directory name. No problems found.

You can try to use fpFLAC from the command-line to check if it works at all, for example: fpFLAC -8 -f <wavfile> <flacfile>
Title: A multicore FLAC encoder
Post by: Glenda on 2011-01-18 10:02:34
The encoder flags errors on every attempt with EAC,  possibly not compatible with the latest version of eac


I've downloaded "eac-1.0beta1.exe" and configured the fpFLAC arguments with -f %s %d -T "ARTIST=%a" -T "TITLE=%t" -T "ALBUM=%g" -T "YEAR=%y" -T "TRACK=%n" -T "GENRE=%m" -8 -V. I've also added spaces in the directory name. No problems found.

You can try to use fpFLAC from the command-line to check if it works at all, for example: fpFLAC -8 -f <wavfile> <flacfile>



Win7x64:

(http://i299.photobucket.com/albums/mm294/regal55/fpFlac.jpg)
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2011-01-18 11:47:42
It seems that the libraries are of mixed type (32 and 64 bit).

Try the following:
1. Uninstall fpStream or delete the folder.
2. Unpack fpStream_1_0_0_23.7z into a folder of your choice. No installation is required.
3. Run 'fpFLAC'

If you haven't installed the Visual C++ redistributables yet, you can download them here (http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=bd512d9e-43c8-4655-81bf-9350143d5867). They are required.
Title: A multicore FLAC encoder
Post by: Glenda on 2011-01-22 02:40:42
If you haven't installed the Visual C++ redistributables yet, you can download them here (http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=bd512d9e-43c8-4655-81bf-9350143d5867). They are required.



Duh,  I missed that part.  Running like a dream now,  compresses so fast all I see is a flicker of the fpFlac.

Thankyou
Title: A multicore FLAC encoder
Post by: seanyseansean on 2011-01-22 12:41:37
I've just noticed this. Good work

1 question though - is it still subject to the same bug as in the Windows version of FLAC - i.e. is it limited to files of around 2GB due to the libraries used?

Reason I ask is because I try to merge all my multi-disc titles into 1 file,  with the standard Windows FLAC I have to split them into around 2/3 disc files.

Thanks again!

p.s. the bug I am on about is referenced here:

http://www.hydrogenaudio.org/forums/index....st&p=483172 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=53872&view=findpost&p=483172)
Title: A multicore FLAC encoder
Post by: lvqcl on 2011-01-22 12:55:13
Quote
is it limited to files of around 2GB due to the libraries used?


1. from the very 1st post:
Quote
- Win64 version: The total WAV file size sum must not exceed 400GiB.


2. take a look at http://www.hydrogenaudio.org/forums/index....st&p=725304 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=84014&view=findpost&p=725304)
Title: A multicore FLAC encoder
Post by: seanyseansean on 2011-01-22 13:10:38
Quote
is it limited to files of around 2GB due to the libraries used?


1. from the very 1st post:
Quote
- Win64 version: The total WAV file size sum must not exceed 400GiB.


2. take a look at http://www.hydrogenaudio.org/forums/index....st&p=725304 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=84014&view=findpost&p=725304)


Can't believe I missed that text in the FP, thanks, but double thanks for the Case version of the original FLAC encoder. You've just saved me a lot of grief and time
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2011-01-22 14:03:13
1 question though - is it still subject to the same bug as in the Windows version of FLAC - i.e. is it limited to files of around 2GB due to the libraries used?

The max supported file size of 4GB is limited by the WAVE format which is the only supported format in fpFLAC right now. Internally, fpFLAC is able to process files up to 500GB.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2011-01-22 14:11:48
Quote
is it limited to files of around 2GB due to the libraries used?


1. from the very 1st post:
Quote
- Win64 version: The total WAV file size sum must not exceed 400GiB.



Did I say 400GiB, not 500? 
I haven't calculated the exact value but this limit is not because of the WAV format. It comes from the internal memory handling (max process virtual memory (2TiB) / number of memory buffers (4 or 5)) .
Title: A multicore FLAC encoder
Post by: Vaako on 2012-09-20 00:53:29
if I use this command line options
Code: [Select]
-8 -f %s %d
and if I do the "Test encoder" or like to Save the settings I receive an error message: Invalid replacement tag found!

What would be a correct command line?
Title: A multicore FLAC encoder
Post by: Vaako on 2012-09-20 01:28:51
BTW, I've decided to add some missing command-line options into fpFLAC (-b, -m, -M, -e, -A, -l, -p, -q, -r and --replay-gain).


Can you list the total command-line options and it's intentions of fpFLAC?
Title: A multicore FLAC encoder
Post by: lvqcl on 2012-09-20 04:09:29
if I use this command line options
Code: [Select]
-8 -f %s %d
and if I do the "Test encoder" or like to Save the settings I receive an error message: Invalid replacement tag found!

What would be a correct command line?


It's a EAC related question.
Title: A multicore FLAC encoder
Post by: GeorgeFP on 2012-09-20 19:18:25
Can you list the total command-line options and it's intentions of fpFLAC?


Just call fpFLAC.exe in the command line to see the options.
Title: A multicore FLAC encoder
Post by: wildstoo on 2013-01-19 23:00:57
Is it normal that fpFLAC creates files with different sizes than standard FLAC with the same compression setting? I'm a noob so feel free to educate me
Title: A multicore FLAC encoder
Post by: Glenda on 2013-10-12 10:40:53
if I use this command line options
Code: [Select]
-8 -f %s %d
and if I do the "Test encoder" or like to Save the settings I receive an error message: Invalid replacement tag found!

What would be a correct command line?


It's a EAC related question.



EAC question or not fpflac.exe many of us can't get fpflac to work with the latest EAC.  Something to do with the way EAC is handing filenames as far as I can tell.  Anyone have a command line that works ?
Title: A multicore FLAC encoder
Post by: Isabelxxx on 2015-05-14 21:25:56
I suposse this one is dead but... this is a copy of the post I put in the cuda encoder.

Quote
Hi Gregory, have been following yours and the fpflac implementation and noted there is one important missing feature (at least for me): the --test option.

Checking an existing flac file like you do with -v option but without the wav requirement and no output, just verifying if the calculated MD5 matches the stored signature.
I suppose in your implementation the idea would be a bit different than the work you have been done up to now since the point is not checking if the output matches the input with your new gpu encoding but making use of the GPU to calculate faster the MD5 hash, independent of the flac "enconding".

Just with that and a batch file you would make an entire 3TB audio database verification a matter of minutes instead of hours.


Gregory said the GPU would not be very good for that task so any possibility to make add it to the multicore enconder?
Title: A multicore FLAC encoder
Post by: slks on 2015-05-16 04:50:54
Is there any advantage to a multicore encoder versus starting a separate encoder on each core (like foobar's converter?)

My experience with multicore encoders (not this particular one) has been that they keep my four cores at 60 or 75% usage, whereas simply starting four single threaded encoders will keep each core 100% utilized.
Title: A multicore FLAC encoder
Post by: Isabelxxx on 2015-05-16 18:40:37
?
The approach is totally different, not sure what are you asking.

A multicore encoder takes 1 file and use X cores in the process.
A multi-instance encoder takes X files and uses X cores.

Both have their advantages and limitations.
Title: A multicore FLAC encoder
Post by: slks on 2015-06-05 10:47:40
I know the approach is different. What I'm asking is what's the advantage?
Title: A multicore FLAC encoder
Post by: saratoga on 2015-06-05 15:50:31
I know the approach is different. What I'm asking is what's the advantage?


Multiple instances is obviously faster if you have multiple tracks, but I guess doesn't help you at all if you only have one track.

Probably the best approach would be to cut an input into multiple tracks, encode them in parallel and then stitch them together.