Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC decoder testbench (Read 22623 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: FLAC decoder testbench

Reply #51
I think these edits are in line with the facts?
Yes, that is OK.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #52
Not completely true though, seems that files 26, 27 and 55 differ. (26 and 27 are not bit-identical files, but same size. 55 is different size.)
I should have put forth the point: could one merely test 60 to 63?


Filled in some fields:
* Amarok. I see you have edited that at some point. Anyway, on this computer, it returns silence (not skipping) on the 20-bit file number 37, and then it certainly also does the same on a 20-bit predictor overflow test too.
* mpv and Mplayer based on old ffmpeg fail the last file. Not sure if it was worth mentioning since it is kinda known ... but so I did.

Re: FLAC decoder testbench

Reply #53
The silence on 20 bit files would be the same reason that Cog failed those files. Arbitrarily only supporting multiples of 8 bits in the code interfacing libFLAC to the player.

It must be really fun where mishandling libFLAC can result in outright freezes or crashes of the player, or worse, crashing the entire embedded OS.

Re: FLAC decoder testbench

Reply #54
Not completely true though, seems that files 26, 27 and 55 differ. (26 and 27 are not bit-identical files, but same size. 55 is different size.)
I already forgot about those. Yes, there were some minor corrections. See https://github.com/ietf-wg-cellar/flac-test-files/pull/3 for details.

Quote
I should have put forth the point: could one merely test 60 to 63?
Yes, I don't think the changes to files 26, 27 and 55 should cause any decoder to fail them where they didn't fail before. The reverse is the case though, as explained in the PR linked above, one player complained about this that weren't strictly according to spec.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #55
Haha, so someone reads specs with a harsher eye than the reference implementation does?

I did some updates, but in case there is a revision 7 ... volunteers, don't bother to do JRiver yet unless you are a paying user. It is only trialware and I couldn't kick it from my computer altogether, so each user who wants to test it, only has 30 days. Also this computer isn't friends with MediaMonkey anymore it seems.

Added VUPlayer, and also: VUPlayer's audiotester.exe verifies them all except complains "EXTRA SAMPLES" on number 45 (not to whine over people who do a job - but if every track had started with speaking out its number, I would have detected skipping without staring at it. Oh, and I have compared the "gritty" the last four seconds of 44 across players.

Re: FLAC decoder testbench

Reply #56
I just uploaded revision 7 of the testbench, see the wiki page for the link: https://wiki.hydrogenaud.io/index.php?title=FLAC_decoder_testbench

Following a bug report by @kode54 it turned out file 63 of r6 was faulty, so this is replaced. Also, file 64 is added, testing rice partition with escape code 0. This works in most decoders, but is not decodeable by libFLAC 1.3.3 (that specific version only) because of a bug.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #57
Files 54 and 55 make foobar2000 v2 beta 6 on 32bit windows 7 force runtime error. They are OK in 1.6.12.

And file 64 is detected as "Unsupported format or corrupted file" and unplayable in both fb2k 1.6.12 and v2 beta 6.



Re: FLAC decoder testbench

Reply #60
Care to do the test then?


Looking back at it, I think the testbench could have included non-subset files: if player P cannot handle file X, is that due to this or that? If it can handle file Y that uses all but one of the features of X and also goes to -l 16 and -r13, that would inform about whether the failures occur due to
* particular features considered unnecessary, like multi-channel playback
* ... or high resolution
* "arbitrary bad design" with no relation to subset bounds other than in coincidences with the previous two items

Re: FLAC decoder testbench

Reply #61
Files 54 and 55 make foobar2000 v2 beta 6 on 32bit windows 7 force runtime error. They are OK in 1.6.12.
Is that with some custom components?

And file 64 is detected as "Unsupported format or corrupted file" and unplayable in both fb2k 1.6.12 and v2 beta 6.
There will be update for 1.6 series later with new libflac. foobar2000 v2.0 beta 7 was just released that now decodes all these test tracks and 32-bit FLACs.


Re: FLAC decoder testbench

Reply #63
Is that with some custom components?
Yes, indeed. Clean installation has no such problem.
I found problematic component. It was foo_multisource. Notice, that problem only occurs with fb2k v2, not 1.6.12.

Re: FLAC decoder testbench

Reply #64
File 64 passed in fb2k v2 beta 7!

 

Re: FLAC decoder testbench

Reply #65
File 64 fails with XMPlay FLAC decoder revision 13.

Re: FLAC decoder testbench

Reply #66
All flac files plays just fine with mpv and ffplay.

Re: FLAC decoder testbench

Reply #67
All flac files plays just fine with mpv and ffplay.
Versions? Do you happen to know what is the earliest ffmpeg version whose ffplay handles everything - since ffmpeg is truly a Big Thing?

Their failures in the current table are with (copy/paste from the margin box note):
* mpv-x86_64-20211031-git-4a80de9 on Windows, testbench revision 5 & files 60-63
* ffmpeg version 4.4.1-essentials_build-www.gyan.dev with testbench revision 5

Also I see that the foobar2000 line is now v2 beta. Given the number of alien cat owners here, I think there at least should be a margin note on how fb2k-stable behaves.

Re: FLAC decoder testbench

Reply #68
Updates:
* MusicBee - shining colours!
* MediaMonkey has improved between 5.01 and 5.04 ... or maybe it is just how it interacts with other hardware

"Non-updates", just verified for other versions:
* foobar2000: files 60 and up verified with stable version 1.6.14
* ffplay: blocksize 16 still is broken as of today's build N-109368-gcc11afe502-20221206.

Re: FLAC decoder testbench

Reply #69
ffplay requires block size at least 128, at least on CDDA.
And its errors are apparently in the player only - not in ffmpeg's "decoder". Nor the encoder.

(1) Initial test: blocksizes 16 and up in steps of 4.
Result: gradually less annoying up to 128, which sounds fine.

(2) Same results when trying all blocksizes 120 to 130 in steps of 1, encoded as follows:
* Using reference flac.exe with both fixed predictors (-2) and various attempted harassments up to -q 15 -l 32 (and some -r that would affect 128 only)
* Using ffmpeg's FLAC encoder, no other options set than -frame_size

(3) Verified the ffmpeg-encodes with flac -t. All fine!

(3) Then used ffmpeg to decode convert files ffplay couldn't play. (That is, convert to ffmpeg's default blocksize.) All fine!


This indicates that ffmpeg encodes and decodes correctly, it is just ffplay that chokes on too low blocksize.
( @mycroft , is ffplay your department?)

Re: FLAC decoder testbench

Reply #70
ffplay is toy and demo app, use mpv always.

Re: FLAC decoder testbench

Reply #71
I wonder where you can decode FLAC encoded on a Ogg container else than Foobar (which decodes FLAC too - which are smaller in size too)
I tried to have these files opened by some old wave editor that decoded .ogg files to see if it improved the usability of the FLAC format backward but couldn't succeed in any.
Hope to have added something for your testing...

In my opinion them work well (haven't gone into the compression algorythm though, but just run some rip/encode/decode/test routines at level 7 of compression and haven't found anything wrong - from library version 2007 (maybe 1.0.4) to 1.4.1)

good vibes

Re: FLAC decoder testbench

Reply #72
Today I tested the FLAC testbench v6 on a Toyota Entune 3.0 unit.  The older 2.0 units only supported MP3, AAC(LC), and WMV.  The new 3.0 units support that plus OGG-Vorbis, hi--res WAV (LPCM), FLAC and ALAC, up to 192/24.

*Anything above 48KHz will be down-sampled to 48KHz/24-bit.

I'll only list what files has issues, anything not listed played fine.

Track 19 - Skip
Track 20 - Skip
Track 35 - Skip
Track 36 - Skip
Track 42 - Skip
Track 43 - Skip
Track 44 - Skip
Track 45 - Plays, but no progress indication.
Track 46 - Skip
Track 51 - Skip
Track 54 - Skip

Re: FLAC decoder testbench

Reply #73
Are you sure you tested revision 6? r7 has been out for a while.

Assuming you did, this is only the second hardware unit tested that has no failures or freezes in the list. So, apparently most hardware implementations have problems with one or more files that are not gracefully handled.
Music: sounds arranged such that they construct feelings.

Re: FLAC decoder testbench

Reply #74
The file I downloaded only has 63 FLAC files in it.  Seems I used the link in post #50, which is a revision of the page that links to v6. I can test again later with v7.