[vorbis-dev] PATCH: utf8 implementation for oggenc (Was: UTF-8 in comments)

[Daniel Resare]

On Sun, Mar 18, 2001 at 11:57:19AM +1100, Michael Smith wrote:
> >
> >Don't bother about that now. Just create that wrapper func/lib/whatever now 
> >so that it's there, and implent UTF-8 conversion for those systems that 
> >support it. Others can just write the tags unconverted like now, for now. 
> >Then, as you find ways to get this working on those other systems, you can 
> >update the wrapper for those systems. This shouldn't be a problem really 
> >since there's already oggs with "normal" text in the tags, so that would have 
> >to be supported anyway..
> 
> Well, plain-ascii text is already UTF-8, since it's a strict subset
> of UTF-8. So for 'normal' text, there's no problem. However, other
> things won't be supported, ever, and shouldn't be. 
> 
> Functionality needs to exist on all platforms, not just some. At the 
> VERY LEAST, all the widely-used unixes and win32 would be required
> before I'd add the code.
> 

Ok, my two months old baby has been nice to me and i've been able to code
away on the UTF-8 problem :) I have implemented a generic utf8 conversion
routine suitable for converting strings before putting them in
comment-fields. My idea is to use the routine as a fallback if iconv() is not
available or don't have the needed features (an iconv implementation can
conform to spec without supporting for example UTF-8) Features:

* portable (no external library is needed, charsets are statially compiled
into the binary)
* simple (~200 lines of code, nothing obfuscated I think)
* compatible licence (GPL, or whatever the vorbis-tools maintainers choose
in the future)

Since this is my first try on actually coding something more than 3 line
patches in C apart from short school exercises some 5 years ago the code
might need some going through and restructuring. Please let me hear your
comments and I will to my best to fix any problems.

I think it would be better to have *.TXT and make_code_map.pl in a
subdirectory, could someone with better automake knowledge please enlighten
me on how to do that?

I include two charsets for testing purposes (new charsets are trivial to
add at a cost of about .5 kb each)

The patch is against vorbis-tools-1.0beta4, since i can't get automake
dependency handling working it might be neccesary to do a 'cd oggenc; make
charsetmap.h' to get it to work. I will be looking into this tomorrow.

cheers/daniel


-- 
nuclear cia fbi spy password code president bomb
8D97 F297 CA0D 8751 D8EB  12B6 6EA6 727F 9B8D EC2A



<HR NOSHADE>
<UL>
<LI>application/x-gzip attachment: oggenc-utf8.diff.gz
</UL>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: oggenc-utf8.diff.gz
Type: application/octet-stream
Size: 9636 bytes
Desc: not available
Url : http://lists.xiph.org/pipermail/vorbis-dev/attachments/20010319/3bf86749/oggenc-utf8.diff.obj