HydrogenAudio

Lossy Audio Compression => Opus => Topic started by: thinktink on 2013-07-16 16:39:37

Title: libopusfile and Unicode names.
Post by: thinktink on 2013-07-16 16:39:37
I'm in a bit of a bind at the moment.  I've developed a Winamp input plugin to play .opus files but I had to do it by converting long file names to the 8dot3 short names so that I could pass them on to the libopusfile library functions that do not support unicode file names.  Little did I know however that people can actually disable 8dot3 file name support.

Apparently, once 8dot3 file name support is disabled on a volume it is impossible for unicode incompatible functions to reference a unicode named file with international characters in the file name.

Or is there an alternative, or if not, who do I contact to get the libopusfile libraries updated to support unicode files?

tia.
Title: libopusfile and Unicode names.
Post by: bryant on 2013-07-16 17:57:27
I have not looked at the API for libopusfile, but does it provide the ability to access files with I/O callbacks rather than using filenames? Libwavpack provides these as an option, and I used them for my winamp plugin.
Title: libopusfile and Unicode names.
Post by: embryonic on 2013-07-16 18:27:07
Looking at the docs for libopusfile, it does seem to.

4.5.2.6 OP_WARN_UNUSED_RESULT OggOpusFile ∗ op_open_callbacks (void ∗_source, const OpusFileCallbacks ∗_cb, const unsigned char ∗_initial_data, size_t _initial_bytes, int ∗_error)
Open a stream using the given set of callbacks to access it.
Title: libopusfile and Unicode names.
Post by: thinktink on 2013-07-16 19:40:04
I thought I set a subscription for instant e-mail notification?!   

Anyways, thanks, I'll look at the callback functions later on.  Although I think the original reason why I didn't use them was because they didn't support multichannel re-multiplexing.  I'll look again.
Title: libopusfile and Unicode names.
Post by: derf_ on 2013-07-31 12:36:51
Anyways, thanks, I'll look at the callback functions later on.  Although I think the original reason why I didn't use them was because they didn't support multichannel re-multiplexing.  I'll look again.


Normal place for opusfile discussion is the mailing list: <http://lists.xiph.org/mailman/listinfo/opus (http://lists.xiph.org/mailman/listinfo/opus)>.

All of the other I/O backends are implemented via the callbacks API, so it supports all the same things.

What needs to be done to add unicode support for Windows?
Title: libopusfile and Unicode names.
Post by: Peter Harris on 2013-07-31 16:31:12
What needs to be done to add unicode support for Windows?


It's a pain.

Windows prefers to operate in UTF16, so a parallel API that takes wchar_t (and calls _wfopen instead of fopen) would be the Windows thing to do, although it messes up the API.

The other alternative is to keep everything in UTF8 and do all the conversions at the last second.

See https://trac.xiph.org/ticket/268 (https://trac.xiph.org/ticket/268) for the vorbisenc patch set.

I suspect "provide an I/O callback interface and let the caller deal with it" is about the best a cross-platform C library can do.
Title: libopusfile and Unicode names.
Post by: derf_ on 2013-07-31 17:40:08
The other alternative is to keep everything in UTF8 and do all the conversions at the last second.

See https://trac.xiph.org/ticket/268 (https://trac.xiph.org/ticket/268) for the vorbisenc patch set.


Thanks for the pointer. Keeping things in UTF-8 would be my preference, as that is, you know, what a sane API would do. Not sure when I'll get to this, but you're right, callbacks would certainly work in the mean time.
Title: libopusfile and Unicode names.
Post by: derf_ on 2013-08-06 22:30:35
Support committed in http://git.xiph.org/?p=opusfile.git;a=comm...;h=f65dab6f653a (http://git.xiph.org/?p=opusfile.git;a=commitdiff;h=f65dab6f653a).
Title: libopusfile and Unicode names.
Post by: thinktink on 2013-08-07 03:55:30
Support committed in http://git.xiph.org/?p=opusfile.git;a=comm...;h=f65dab6f653a (http://git.xiph.org/?p=opusfile.git;a=commitdiff;h=f65dab6f653a).


Looks nice.

Two things however (1 question and 1 statement.)

Question:
I dropped that utf16_to_utf8 function into a dummy project and compared it to ::WideCharToMultiByte(CP_UTF8,...); and so far all of the Unicode strings I have compared between the two have exactly matched.  Would it be safe to assume that if this gets implemented in the next compiled and official release of the libopusfile library that all I will need to do is to convert the filename by a call to ::WideCharToMultiByte(CP_UTF8,...)?  I'm not uber familiar with unicode strings, utf8, utf16, or whatnot and just want to confirm.

Statement:
Me personally, as an active Windows developer, I would prefer a separate op_open_fileW (while retaining the above changed functionality on the original op_open_file function) since right now all my internal string references to filenames are all unicode anyway and maybe possibly save time and resources having to convert filenames all the time.  At the moment, I have internally implemented an op_open_fileW function which uses op_open_callbacks although I'd prefer native support so that I get consistency.  So far it's working though.
Title: libopusfile and Unicode names.
Post by: derf_ on 2013-08-10 01:27:04
Would it be safe to assume that if this gets implemented in the next compiled and official release of the libopusfile library that all I will need to do is to convert the filename by a call to ::WideCharToMultiByte(CP_UTF8,...)?


Yes, that should work fine.

Me personally, as an active Windows developer, I would prefer a separate op_open_fileW (while retaining the above changed functionality on the original op_open_file function)


The problem is that then I have to provide this UTF-16<->local character set conversion for all other platforms. There are a number of reasons why this is a pain to do, and it provides no benefit, since unlike Windows, other platforms can generally access all of the files on the disk using char strings, without resorting to wchar_t.