[vorbis] Re: UTF8, vorbiscomment, oggenc, and 'vcedit.c'

[Glenn Maynard]

On Fri, Jan 11, 2002 at 06:58:28PM -0500, Peter Harris wrote:
> > > umlaut-u in wchar_t *argv[] appears as superscript-n when wprintf()ed
> >
> > That seems to mean they're doing some weird conversion that we can't
> > really fix.  It might be worth adding a @file option (read arguments),
> > which is always in UTF-8, so that scripts that want to interface with
> > this always have at least one reliable way of getting these characters
> > through.  (Or perhaps in the locale/codepage encoding, I'm not sure.)
> 
> We can detect UNICODE via the magic word 'FEFF' (looks like FFFE in
> little-endian files; looks like EFBBBF in UTF-8) and use that. In the

Aack!  That's a Windowsism; requiring it in anything but Windows will
annoy a *lot* of people, myself included.

How about this: read it in the current locale.  If a program (ie a
script) wants to call it with UTF-8 data, it can change the locale to
UTF-8 for that call.  That way, it works like every other modern locale-
sensitive package, and isn't an exception.

For Windows, you could get away with using FEFF to change to Unicode,
but then you have fairly different behavior between Windows and Unix.
That might be inevitable, though.

> absence of the UNICODE magic word, we can default to the ANSI code page (At
> least notepad seems to save files in the ACP. Which makes sense: notepad
> doesn't have a console to get the console code page from. On the other hand,
> what if people use a console app to edit @file? Aarrgh!)

I think that if the file isn't in the ACP, we shouldn't worry; we have
no (real) way of telling what it is.  (What if I edit it in Big5 then
change locales/codepages?  I wouldn't worry about that case, since
you're not supposed to do that anyway.)

> Check out my patched oggenc.exe (see other message) and let me know what you
> think.

I'll look at it, but far more important is if it works in 9x, and I
don't have one of those--and not a Japanese one.


-- 
Glenn Maynard

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.