Page Index Toggle Pages: 1 [2] 
Topic Tools
Hot Topic (More than 10 Replies) UTF-8 support (Read 6,058 times)
Captain John
Ex Member


Re: UTF-8 support
Reply #1 - Jan 29th, 2011 at 5:13pm
Post Tools
Michael Prager wrote on Jan 29th, 2011 at 5:06pm:
That's because the data was saved to the database in KOI8-U while it was loaded in Latin1.

  Isn't this the cure ?  Writing english to UTF http://www.yabbforum.com/community/YaBB.pl?num=1284419291/15#15
  
Back to top
 
IP Logged
 
Michael Prager
Boardmod Team
Development Team
*****
Offline



Posts: 975
Location: Germany
UTF-8 support
Jan 29th, 2011 at 5:06pm
Post Tools
I've familiarized myself a bit with the internals of Unicode and the UTF-8 encoding. Good references on the subject I've found:

http://www.unicode.org/versions/Unicode5.2.0/
http://www.cl.cam.ac.uk/~mgk25/unicode.html
http://perldoc.perl.org/perluniintro.html
http://perldoc.perl.org/utf8.html
http://perldoc.perl.org/perlunitut.html
http://perldoc.perl.org/perlunicode.html

Looks like UTF-8 is the best choice to me, as it is fully supported by modern browsers as well as perl (as of 5.8) and by mysql (5.0). So we can encode everything in it and bother no more about encoding. Since it is compatible with latin1 it won't have much impact on disk usage for english based forums.

I've played a little with the current code and was shocked how easy it was to get working results: replace $yycharset with "UTF-8" in Languages/English/Main.pl and create a database with utf8 default encoding. That's it. Shocked

But this also reveals a serious problem with current multi language support in YaBB:
the encoding of data written to and read from the database (no matter if mysql or flatfile) is always dependent on the current user's selected language! So if I have two installed languages for my forum (for example one English and one Russian) and one user posts something in Russian language, every other user that uses english language setting will only see corrupted data! That's because the data was saved to the database in KOI8-U while it was loaded in Latin1. So any YaBB forum out there using multi language will probably have mixed or broken encodings Undecided
  

Nail here for a new monitor! --> [x]
Back to top
WWWICQ  
IP Logged
 
Page Index Toggle Pages: 1 [2] 
Topic Tools
 
  « Board Index ‹ Board  ^Top