Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC v1.4.1 (Release) (Read 19470 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: FLAC v1.4.1 (Release)

Reply #75 –
Quote from: Porcus on
My problem here is the attitude that the problem is irrelevant because it only occurs in the Windows shell
I never said it was irrelevant. I just wanted to stress that this is a problem that FLAC is supposed to fix because Windows does something stupid. You say FLAC does stupid things by outputting the same stuff over 100 times when you feed it nonsense arguments, but I don't hear anyone on why the terminal output is so slow. Honestly, why does rendering 25 characters take as much time as processing 80.000 samples?

I think it is because the terminal deliberately slows down output to emulate a real terminal, with a certain limited baudrate. I don't understand why Powershell would do that too though.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.4.1 (Release)

Reply #76 –
Windows does stupid things yes, we've known that since forever. And unless you can expect it to change its ways soon, an application for Windows - including one developed for cross platform - has to, in some way or another, adapt to that to get around reasonably.

And sure I can provoke bad behaviour by nonsense arguments, but "-5" is not a nonsense argument. And those who want to use --fast probably don't want thirty percent performance hit either. (Though I'd argue it would be a bit hasty to make --fast synonymous to -0s. There are easy fixes like writing "completed: 23%", then "completed" does not need to be overwritten, for example, but again ... better do it properly.)

Quote from: ktf on
I don't understand why Powershell would do that too though.
Hunch: Because they run in the same Windows console?

Re: FLAC v1.4.1 (Release)

Reply #77 –
The "alternative shells" may simply skip repeated strings I suppose?

I can understand the use of backspace (wv uses it too), as '\r' and '\n' can have different meanings in different platforms. For Windows command prompt '\r' is the fastest way to move the cursor to the beginning of a line without starting a new line.

If the progress display is going to be changed, it can be something like this:
Code: [Select]
Ratio  Completed
1.000  100%
Much fewer backspaces to use, and of course, only update when the values are actually changed. Personally I prefer more precision on ratio, 1.00000 would be more useful for benchmarking higher compression settings.

Re: FLAC v1.4.1 (Release)

Reply #78 –
Quote from: bennetng on
The "alternative shells" may simply skip repeated strings I suppose?
I think that what those shells do, is receive the string, tell the OS that "sure, got it!" and then the process keeps running while the shell turns to the user and outputs it - but the Windows terminal OTOH, receives the string, and then the OS stands waiting while the shell outputs it and returns "OK, message sent" to the OS. Hence the delay.

Quote from: bennetng on
If the progress display is going to be changed, it can be something like this:
Code: [Select]
Ratio  Completed
1.000  100%
That would take more lines. FLAC seems to attempt at one line per file if it doesn't become too long - although it isn't always good at delivering parseable output (ticket here).
If one were to overhaul it, I guess it could be like
inputfilename.aiff is 123456789 bytes, compressing at ratio 0.86;  60% done
or with stdin or --ignore-chunk-sizes
<filename or whatever> is unknown size, compressing at ratio 0.86;  40 MB done
(I used a double space after semicolon to allow for three-figure MB size, that gives nine characters to update - but the 0.86 need not be updated very often either. Problem about the latter: is the 40 to be output or input size? I don't think it is that important.)

Also upon decoding, it seems to overwrite "complete" all the time. But it doesn't update that often, it knows that decoding is fast.

I guess FLAC users will object to losing information, but TAK does it as simple as one dot per ten percentage points or so.

Quote from: bennetng on
Personally I prefer more precision on ratio, 1.00000 would be more useful for benchmarking higher compression settings.
I agree, but that precise information is not necessary until done. When done it could report e.g. 0.86 (-13.9%) and with higher precision indicated as 1.00 (-0.04%).

Re: FLAC v1.4.1 (Release)

Reply #80 –
Quote from: Porcus on
My problem here is the attitude that the problem is irrelevant because it only occurs in the Windows shell - as if anyone could ever expect Windows CLI application to be run in a Windows shell, those who run command-line on MS-Windows must be expected to use something else even without being told.

To be fair, I think that's mostly my fault rather than ktf's. Perhaps I shouldn't have butted in, especially considering I have absolutely nothing to do with FLAC development anyway.

Re: FLAC v1.4.1 (Release)

Reply #81 –
Quote from: ktf on
Okay, it seemed easier to fix (and with less strings attached) than I anticipated.

Proposed fix: https://github.com/xiph/flac/pull/492

Windows build attached.
Much better, thanks.

PS H:\> measure-command {h:\flac -0fpb128 image.wav -o 128.flac}|select totalseconds
image.wav: wrote 3067469939 bytes, ratio=0.714
TotalSeconds
------------
  30.1329976


PS H:\> measure-command {h:\flac -0fpsb128 image.wav -o 128.flac}|select totalseconds
TotalSeconds
------------
   29.626285


PS H:\> measure-command {h:\flac -t 128.flac}|select totalseconds
128.flac: ok
TotalSeconds
------------
  21.9606564


PS H:\> measure-command {h:\flac -ts 128.flac}|select totalseconds
TotalSeconds
------------
  21.5206184

Re: FLAC v1.4.1 (Release)

Reply #82 –
Slowdown problem solved! :-D

But I managed to provoke the following - oddly enough, only in PowerShell (when it occurred, I tried to give cmd in the same window and the same flac command, and the error did not reproduce). Note the zero percents and the "c" following the one-character-shorter .wav files:
Code: [Select]
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (2).flac: 0%
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (2).wav: 0% c
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (3).flac: 0%
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (4).flac: 0%
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (4).wav: 0% c
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (5).flac: 0%
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (5).wav: 0% c
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (6).flac: 0%
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy.flac: 0% comp
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy.wav: 0% compl
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven.flac: 0% complete, rwrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven.wav: 0% complete, rawrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesFullOfDingbats➀ ➁ ➂ ➃ ➄ ➅ ➆ ➇ ➈ ➉ ➊ ➋ ➌ ➍ ➎ ➏AndAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (6).flac: wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesFullOfDingbats➀ ➁ ➂ ➃ ➄ ➅ ➆ ➇ ➈ ➉ ➊ ➋ ➌ ➍ ➎ ➏AndAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (6).wav: wrote 68998568 bytes, ratio=0,498

Resizing the window slightly, I think the problem is a combination of 0% (or maybe single-digit percent?!?) and linebreaks:
Code: [Select]
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (2).flac: 0% complete, ratio
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (2).wav: 0% complete, ratio=
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (3).flac: 0% complete, ratio
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (4).flac: 0% complete, ratio
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (4).wav: 0% complete, ratio=
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (5).flac: 0% complete, ratio
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (5).wav: 0% complete, ratio=
wrote 68998568 bytes, ratio=0,498
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (6).flac: 0% complete, ratio
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy.flac: 0% complete, ratio=4,4
wrote 68998568 bytes, ratio=1,000
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy.wav: 0% complete, ratio=2,22
wrote 68998568 bytes, ratio=0,498

... another minor resize and I get it in a different track:
Code: [Select]
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (10).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (11).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (12).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (13).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (14).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (15).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (16).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (17).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (18).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (19).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (2).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (20).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (21).flac: 0% comwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (3).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (4).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (5).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (6).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (7).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (8).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy (9).flac: 0% compwrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents - Copy.flac: 0% completewrote 4015735 bytes, ratio=1,000
Anal Trump {2016} That Makes Me Smart! - 06 - Poor People Are Too Stupid To Get A Loan From Their Parents.flac: 0% complete, ratiowrote 4015735 bytes, ratio=1,000

Re: FLAC v1.4.1 (Release)

Reply #83 –
Can't reproduce. When completed, "100% complete," always disappears, only "wrote x bytes, ratio=y" remains. Could be locale, font etc specific.

Click the icon on top-left corner, "Default" is global, affects everything in all consoles, "Properties" is specific to the current app.

If it is related to wrap text, it is controlled by Windows, not individual apps.

Re: FLAC v1.4.1 (Release)

Reply #84 –
Would it be possible to add an option to hide the banner and have only the progress bar?  Similar to ffmpeg's -hide_banner option. 

Re: FLAC v1.4.1 (Release)

Reply #85 –
Quote from: bennetng on
Can't reproduce. When completed, "100% complete," always disappears, only "wrote x bytes, ratio=y" remains.
"Can't reproduce" is not far from the situation here - after seeing it once it took me quite some effort to get it to happen on different files.
Unlucky find I guess. But still a find.

Re: FLAC v1.4.1 (Release)

Reply #86 –
Found the following with official 1.4.1 as well - and in the standard cmd shell. (This on a different computer.)
Also, it occurred with a double-figure percentage too - so three hypotheses to be deleted right off.

Since it is not a freshly introduced issue, then it is probably related to the other ways where output isn't completely sanitized.


Quote from: Porcus on
Slowdown problem solved! :-D

But I managed to provoke the following - oddly enough, only in PowerShell (when it occurred, I tried to give cmd in the same window and the same flac command, and the error did not reproduce). Note the zero percents and the "c" following the one-character-shorter .wav files:
Code: [Select]
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (2).flac: 0%
wrote 68998568 bytes, ratio=1,000
[...]

Re: FLAC v1.4.1 (Release)

Reply #87 –
Just for my information, sometimes assertions in libflac come after fields are accessed at least for accesses that amount to convenience renames. Is this okay only because they are renames that all compilers will have long undone, is it only possible when variables are const? Sorry for the generic programming question, I've never come across an ordering like this and would have assumed it would trip up something but it obviously hasn't.

Code: [Select]
FLAC_API FLAC__bool FLAC__stream_encoder_process_interleaved(FLAC__StreamEncoder *encoder, const FLAC__int32 buffer[], uint32_t samples)
{
	uint32_t i, j, k, channel;
	const uint32_t channels = encoder->protected_->channels, blocksize = encoder->protected_->blocksize, bps = encoder->protected_->bits_per_sample;
	const FLAC__int32 sample_max = INT32_MAX >> (32 - encoder->protected_->bits_per_sample);
	const FLAC__int32 sample_min = INT32_MIN >> (32 - encoder->protected_->bits_per_sample);

	FLAC__ASSERT(0 != encoder);
	FLAC__ASSERT(0 != encoder->private_);
	FLAC__ASSERT(0 != encoder->protected_);

Re: FLAC v1.4.1 (Release)

Reply #88 –
Quote from: cid42 on
Just for my information, sometimes assertions in libflac come after fields are accessed at least for accesses that amount to convenience renames.
The example you provide is indeed not the way this is supposed to work. The first and last assert will never trigger, because if these are true, there will have been a null dereference in the assignments.

Anyway, it is important to consider what function assertions serve. They are only checked when debugging is enabled, they help pin down usage or behavior unforeseen when the code was written. They also serve as documentation, especially for the public API. See for example the API documentation, assertions are listed there (example)

Also, fuzz testing with sanitizers has mostly taken over the function assertions like these provide: null dereferences are usually quickly found with proper fuzz testing.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.4.1 (Release)

Reply #89 –
Thanks, I understand it wasn't really a problem and was just curious if compiler magic was being intentionally used to simplify the way it was written. Would have been nice if that could be relied upon.
 

Re: FLAC v1.4.1 (Release)

Reply #90 –
This thing here ...

The console does strange things when it "would want to" have a linebreak. Seems that in more recent Windows versions it tolerates wider windows - but, depending on shell and characters, breaks it up and sometimes inserts whitespaces in the output, and I have a hunch that <BS> simply doesn't work across those.

Depending on shell and characters:
First, the prompt looks different in PS and cmd, and if they don't have the same number of characters, things will happen different places.
Second, character handling is not the same from Windows to Windows (localization), from shell to shell (Powershell outputs ' as '' because ...), and who knows if it depends on the font used.

So my hunch about this line-breaking behaviour is that it cannot really be avoided without diving into Windows specifics.
Meaning, forget about it at least until one decides to start working on flac's screen output more thoroughly, if ever.


Quote from: Porcus on
Found the following with official 1.4.1 as well - and in the standard cmd shell. (This on a different computer.)
Also, it occurred with a double-figure percentage too - so three hypotheses to be deleted right off.

Since it is not a freshly introduced issue, then it is probably related to the other ways where output isn't completely sanitized.


Quote from: Porcus on
Slowdown problem solved! :-D

But I managed to provoke the following - oddly enough, only in PowerShell (when it occurred, I tried to give cmd in the same window and the same flac command, and the error did not reproduce). Note the zero percents and the "c" following the one-character-shorter .wav files:
Code: [Select]
LetUsSeeWhatHappensWhenTheFilenameBecomesAsLongAsThisThisThisLongLongLongLongerLongerEvenLonger11!!!!!eleventyeleven - Copy (2).flac: 0%
wrote 68998568 bytes, ratio=1,000
[...]