Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC decoder testbench (Read 31919 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: FLAC decoder testbench

Reply #75 –
Only file 63 differs, and file 64 has been added between revision 6 and revision 7. Anyway, thanks for testing!
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #76 –
Tested files 63 and 64 from v7.  They play fine.

Re: FLAC decoder testbench

Reply #77 –
Added to the table. Interpreting "will be down-sampled" as not being part of the decoder (rather subsequent processing) I have not remarked those. Could be filled in "Remarks" if one wants to.

Re: FLAC decoder testbench

Reply #78 –
Quote from: Porcus on
Added to the table. Interpreting "will be down-sampled" as not being part of the decoder (rather subsequent processing) I have not remarked those. Could be filled in "Remarks" if one wants to.
BTW, ability to correctly play files 19-21 depends of device/program to do resampling rather than ability to correctly decode FLAC

Re: FLAC decoder testbench

Reply #79 –
Yes, but it would be nice to recognize during parsing of the FLAC file already that the device is unable to render a file properly. In other words, a file with a samplerate that a device cannot play should be gracefully skipped, informing the user the file is unplayable, instead of mangling or even crashing.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #80 –
I am not sure if resampling is "mangling" when it comes to a hardware player. To pinpoint it: what if the device doesn't even have any digital output? If it decodes the FLAC file then it decodes the FLAC file, even if what the device outputs is analog-only.
Even if an in-car player has a digital connection, how many in-car DACs support sampling rates of 54321 kHz? I'd say it is fair game for the purpose to resample to 48000 or 44100.

Re: FLAC decoder testbench

Reply #81 –
With mangling I actually meant playback at wrong speed, not resampling.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #82 –
54321 is still not as malicious as 76543 (prime). SSRC in foobar supports 54321 but not 76543, setting the output to 76543 the result will be an untouched file with no warning or error. RetroArch and foo_dsp_resampler for example have no issue with 76543.

In the case of hardware resampler chips there are also limitations on maximum resampling ratios. For example Cirrus Logic CS8421, CS8422 and TI SRC4192, SRC4193. I have a very old Roland synth/interface with a very old CS8420.

Re: FLAC decoder testbench

Reply #83 –
I was able to test a Mazda Connect unit today in a 2024 CX-90.  I'm not sure if Mazda Connect has generations like the Toyota Entune I tested. (Entune 2.0, 3.0 etc)

This unit is a Panasonic unit, running Mazda Connect software version 10010 (https://car.panasonic.jp/oss/h04lmm2/)

Tested with revision 7 of the Flac testbench. (64 tracks)

All files played! 

The only minor issue I experienced is when track 27 finishes, it doesn't automatically move onto the next track.  The unit doesn't appear to hang, but does require me to manually advance to the next track.


Edit:  I'll see if I can find anything in the manual for any limitations, like if any tracks get down sampled...

Re: FLAC decoder testbench

Reply #84 –
I've added your results to the wikipage. Thanks! Nice to see a hardware device has been found that really does playback all files properly.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #85 –
I couldn't find anything in the manual, or online,  about what codecs are supported or any limitations of those codecs.  Looking through the licenses, the unit looks to be ARM based, likely a Linux based OS.  Seems to be using gstreamer for decoding audio files.  I thinks there's support for ALAC as well.

Re: FLAC decoder testbench

Reply #86 –
Hello. I got bored and decided to test mpv v0.38.0-633-ge509ec0a (or latest git release at the time of writing this) on Windows. I used the revision 7 files linked on the page. Everything passed including the previously failed extreme metadata section. Track 27 caused some odd UI issues (the time bar was jumping around a ton) but the file was able to play just fine.

If someone could add the current results to the page I would be very grateful. I would do it myself but I cannot figure out how to sign into the wiki. I also plan on doing tests for mpv on Android and Linux when I get a chance.

Thank you and have a great day

Edit: Android testing on v0.38.0-432-g112fa549be is done. All files pass and play perfectly. Same weird time bar behavior on track 27 but once again played just fine. Linux testing will come tonight.

Re: FLAC decoder testbench

Reply #87 –
Thanks, I just updated the table.

Quote from: Rucker on
Track 27 caused some odd UI issues (the time bar was jumping around a ton) but the file was able to play just fine.
I haven't marked that in the table, because I'm not considering it a failure. The problem is, that file is made according to an old version of the FLAC specification, and I don't consider strictly following the new spec (without considering that change in 2007) to be a failure. See here for details on this change. The reason I included the file in the testbench anyway, is because some linux distro's (like Debian) have a FLAC encoder in their repositories (flake) that is so old, it actually still creates these files.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #88 –
Quote from: Porcus on
(By the way: care to run the FLAC testbench with the DAW software?)
Audacity is freeware, Reaper has free trial so others can try them if interested.
Here is Adobe Audition 1.5 with this 3rd party plugin:
http://www.vuplayer.com/audition.php

Faulty folder
crash: 04
reject: 06, 11
All other files opened normally as if there is no problem at all.

Subset folder
reject: 22, 37, 62
silence: 45

Uncommon folder
silence: 01, 02, 03, 04
reject: 05, 07, 10, 11

Quote from: Porcus on
@ktf : How do you propose that such software be entered in the testbench table? After all, "doing playback" isn't their primary function.
Pretend it is, and just check whether they can decode to appropriate output?
Anyway, one may consider a separate section, rather than put them into the alphabet with software players?
One key difference between "player" and "DAW" is the ability to export files. foobar2000 for example can export files as well as playback, so applies to both categories. Consider a situation like this:
Quote from: bryant on
So my idea for handling these float-derived integer files is to create a new, smaller bitstream for the “completion” data with a new metadata ID. Old decoders will simply ignore it and consider the file as “lossy” (although we’re still talking less noise than the best DACs). Updated decoders will recognize the new ID and do the lossless decoding. Rockbox never did anything with that info anyway, so it will be unaffected (what’s Rockbox going to do with more than 24 bits...I think it is 16-bit internally).
So Rockbox is considered "pass" as a "player", but for "DAW" I would consider a 16-bit only export "fail".

As for the Audition test above, I indeed exported the decoded files as .wav and verified bit-perfectness of audio data, except for the multichannel files because Audition import these files as separate mono tracks and it does not support export to a single multichannel .wav file.

Re: FLAC decoder testbench

Reply #89 –
Quote from: bennetng on
Here is Adobe Audition 1.5 with this 3rd party plugin:
http://www.vuplayer.com/audition.php
Thanks!

Quote
One key difference between "player" and "DAW" is the ability to export files. foobar2000 for example can export files as well as playback, so applies to both categories.
Yes, but this testbench only applies to decoding, because FLAC has quite a few features that aren't used very often, so this testbench tries a lot of them, causing players to crash. On encoding, there is nothing similar, because the program itself can simply choose to ignore those features. Of course a DAW should support 16-bit and 24-bit exporting and all samplerates, but it only needs to support one blocksize for example, or not support wasted bits. It could even choose to ignore LPC completely and only use fixed subframes (like the flac command line tool does with presets -0, -1 and -2).

That is why decoders are way more interesting to test this way than encoders are.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #90 –
Quote from: ktf on
That is why decoders are way more interesting to test this way than encoders are.
Quote from: bennetng on
As for the Audition test above, I indeed exported the decoded files as .wav and verified bit-perfectness of audio data, except for the multichannel files because Audition import these files as separate mono tracks and it does not support export to a single multichannel .wav file.
To be clear, I was not testing the flac encoder because the Audition plugin was released in 2007 which is very old. I meant, for "player" the audio data are supposed to eventually output as sound and to the ears of listeners, and it is perfectly normal that listeners cannot hear more than 16-bit. For file export it is a different story. For example, even if Audition 1.5 does not support multichannel .wav file output and does not support certain metadata features, the audio data themselves should be bit-perfect for each individual mono channel up to 24-bit (Audition 1.5 uses 32-bit float internal format, so it is lossy for int32 export). Yes I understood you mentioned several times that the flac license mentioned there is no warranty, no guarantee of anything, but because bit-perfect audio is what typical users expect from a lossless codec, if a specific software can't do this it should be clearly mentioned.

So basically, what I talked about is people who submit a compatibility report (like me) for "DAW" applications should perform file export tests to verify for bit-perfectness, not just casually listen to them or simply look at the waveform. This step can also find other potential issues like minor length truncation and so on.

Re: FLAC decoder testbench

Reply #91 –
So, conflicting points:
* If a software application does lossy treatment - for example by importing 32-bit integer to 32-bit float - that is "bad" even if decodes correctly.
* On the other hand, a hardware player that provides analog-only, necessarily does it in a "lossy" way. Also, whether it internally works in 16 or 24 bits before outputting analog, is hardly an issue?

Decimating 24 bits to 16 bits and exporting it as file, is IMHO worse than decimating to 16 and DACing to deliver analog-only. Even if the latter is another lossy step - it breaks no promises.

How to reflect that in the table? Possible suggestion (that is, maybe not even at "suggestion" level yet):
* If we know software handles playback and everything else it offers (export, verification, tagging), then "Plays++" on green background rather than "Plays" on lime. ffplay gets channels allocated wrong? That stays "Plays" on lime with the comment.
* Hardware players: kept as they are, nothing in the top of the table will ever get "Plays++" on green. Want to nitpick on Rockbox because a power user can provoke lossy digital out? Comment it as appropriate, but it still "Plays".

Re: FLAC decoder testbench

Reply #92 –
Another thing is DAWs are in general, quite complicated. Some may by default apply dither to file export, some may apply a surround paw law that changes the level of mulitichannel export, some may import audio files as clip items and apply some 1-2ms of fade in and out. So volunteers who submit results should be careful with these things and make sure all kinds of additional processing are disabled.