Bug: $left() and $right() do not handle DBCS

2006-08-12 09:43:00

I noticed a problem with my text getting chopped off awhile back but it took me time to figure out exactly what was causing it. I have experienced identical behavior in columnsui playlists, trackinfo panels, and foo_browser panels.

The functions $left() and $right() do not process double-byte characters the same way as other functions, such as $substr().

For this example I will use the string 日本語 (Double-byte SJIS encoded "nihongo"). But in reality I discovered the bug while passing various forms of %title%. Lets consider the function:

$left(日本語,2)

The expected output would be the first two characters 日本 (nihon) however the actual output is just the first character 日 (ni). If we change the 2 to a 4 to accomodate for double-byte then the output is as orginally expected. Similarly if we consider the function:

$right(日本語,1)

The expected output would be the last character 語 (go) however the actual output is half of the last character which renders as nothing. If we change the 1 to a 2 to accomodate for double-byte then the output is as orignally expected.

In my experience with working with DBCS encodings I have come accross similar behaviors in other programs and would understand if I needed to code up some extra code to accomodate DBCS. However, every other function of foobar that I have tested did not need this. Lets us consider the function $len().

$len(日本語) outputs a 3 ... ni/hon/go is three characters. this function acts as expected and desired.

As a workaround I have converted my $left() and $right() functions to the following $substr() equivilants:

$substr(日本語,1,2) outputs the expected 日本 "nihon"
$substr(日本語,1,1) outputs the expected 語 "go"

This workaround further demonstrates that other functions in foobar treat double-byte characters properly. However, this workaround can become extremely complicated when used in reality, especially when one must accomodate for text that does or does not exist. IMO the original functions should treat double-byte text just like any other function.

In my actual code I had to convert:
$if(%title%,$trim($left(%title%,$sub($strrchr(%title%,$char(40)),1))),)

into:
$if(%title%,$trim($substr(%title%,1,$if2($sub($strrchr(%title%,$char(40)),1),$len(%title%)))),)

I tried to keep these examples as simple as possible but I think you can see how the workaround can quickly make your code grow significantly. If anyone can provide a better workaround or input as to any settings I may have overlooked that would impact this behavior, please reply.

This problem only affects double-byte text and the majority of foobar users do not use double-byte text. I have also discovered and explained a viable workaround so I would not consider this bug critical by any means. However, as more non-western users begin using foobar I can forsee the issue arising more and more often, and not just with asian encodings. I would not expect the average user to have enough of a grasp of TAGZ and text encodings to understand and quickly formulate a workaround so attention to correcting this bug would be appreciated. I have long been impressed with foobar's ability to handle multiple languages.. such as korean, chinese, japanese, unicode and western encodings all seamlessly on the same page at the same time. Thank you for your time reading my lengthy bug report.