[vorbis-dev] UTF-8 & Hebrew

[Beni Cherniavksy]

On 2002-03-11, Ross Levis wrote:

> Beni Cherniavksy wrote:
> > I thought WinVorbis was supposed to support Unicode
> > but this seems to indicate that it does use unicode when displaying.
> > OTOH, it does read/write UTF-8 correctly.
>
> I presume you mean "does NOT use unicode".  I know very little about
> unicode or UTF-8.  I added a UTF-8 decode function to WinVorbis but
> obviously that is not enough.  I suspect I may have to store the UTF-8
> decoded string in a 16-bit WideString, rather than an 8-bit AnsiString.
> I'll have a go at that in the next day or 2.
>
Oh.  Decoding UTF-8 to 8-bit is a bad idea.  It means that you lose
infromation when reading a tag not representable in the computer's ansi
code page.  That means you will not be able to display such a tag.

You should use GUI widgets that support unicode and use internally the
encoding they work with (16 bit most probably on windows).

Note that unless vorbiscomment is modified to take raw UTF-8 (or unless
you write the files directly), you will not be able to write comments with
characters outside the ansi code page.  To this end, you should either bug
the developers to add raw UTF-8 input/output to vorbiscoment (I'm all for
this, currently vorbiscomment is round-trip unsafe, with no workarounds,
which makes it useless in scripts), add it yourself or at least detect
when you are about to lose info (convert 16-bit -> 8-bit ansi -> 16 bit
and compare) and pop a message box asking the user for confirmation...


-- 
Beni Cherniavsky <cben at tx.technion.ac.il>
                 (also scben at t2 in Technion)
MSVC in default mode doesn't define __STDC__ (correctly).

<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.