[vorbis-dev] UTF-8 & Hebrew

[Beni Cherniavksy]

[took long since at the middle of composing I noticed that I'm late for an
exam, so I had to run - now back to the important things... ;-]

On 2002-03-07, Beni Cherniavksy wrote:

> On 2002-03-07, Ross Levis wrote:
>
> > This user had Russian as his default codepage & couldn't see the Hebrew.
> >  Maybe he needs to swap that around.  I'm not sure what consequences
> > that will have on other apps.
> >
> > Beni, could you try WinVorbis to see if Hebrew appeares OK for you.
> > http://winvorbis.stationplaylist.com
> >
> Will now try, downloading oggenc and VorbisExt to create a file (not at
> home now)...
>
OK, I think I tracked things down...  He probably has not installed the
hebrew loacle correctly.  Or maybe it's something else...

The point is that this system I'm now on is WinNT 4 with Hebrew as default
locale and without administrator rights for me ;-(.  I can't choose a
russian layout for the keyboard, probably something with the russian
installation is strange.  Word shows russian all right, so do NS, MSIE...
However I can't paste from charmap to neither VCE, WinVorbis or PP's
plugin, which are all supposed to use Unicode.  (I can't try VorbisExt
without admin rights, so I revert to VCE).  It shows Hebrew perfectly in
all 3 of them.  However it doesn't show Russian on either of them!  They
all show Russian as "aaaaaa".  Non-unicode apps show Russian as "??????"
(e.g. VCE-95 or .  Also behind the "aaaaaa" is true Russian data that I
can copy from one comment and paste into another!

vorbiscomment and oggenc try to use current locale so they can neither
read nor write Russian comments (vorbiscomment shows Hebrew correctly, if
I choose Lucida Console for the dos prompt).  ogginfo is nicer by
outputting raw UTF-8 (from which I see that my test files are OK).

<rant> Why doesn't vorbiscomment on windows just read/write UTF-8?  At
least add an option for that?  Using the console's pages is almost a lost
case on most windozes anyway, so at least allow wrappers to work and make
the round-trip safe... </rant>

So WinVorbis works fine with Hebrew but can't write Russian.  It reads it
OK (since it bypasses vorbiscomment on reading) but can't show it.  It
shows up as "aaaaaa".  I managed to write Russian comments by:
1. manualy editing the .ogg (which breaks it's CRC).  I put in exactly the
   characters as saved in "Encoded Text" -> UTF-8 from Word.
2. opening with WinVorbis (which was the only one agreeing to read it -
   ogginfo, vorbiscomment, VCE and PP's plugin use libogg - maybe there
   should be an API to tell libogg "trust me, assume the CRC was all
   right"?)
3. copying from WinVorbis' field into peter's plugin (on a non-broken
   version of the file) and saving.

Summary: Hebrew works fine, I can't see Russian on any comment editor,
probably due to mis-installed windows support for russian, but it's there.
Native comment reading/writing tools can operate on it normally,
vorbiscomment (and tools using it) ruins it.

> > Beni Cherniavksy wrote:
> >
> > >I can see all text (Hebrew & Russian, Hebrew is my default
> > >codepage) in Peter's plugin - when I double click the tag to edit.
> > >
I surely do at my home system.  But there I installed both hebrew and
russian support properly.


-- 
Beni Cherniavsky <cben at tx.technion.ac.il>
                 (also scben at t2 in Technion)
Common Lisp is better than Common Source and
Open Source is better than Open Collector (YMMV).

<p><p><p><p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.