Page 3 of 4

Re: Russian language support

Posted: Mon Jan 08, 2007 2:32 pm
by tr808
Phelan wrote:When you take a look at http://gameforge.de then you will see that we have games in quite a few countries. Russia is included as well.
However, it is too soon to talk about release plans in other countries.

i dont think he means a total client in Russian, he simply wants the Rusian alphabet to work, so they can type with eachother in team or clanchat in Russian.

Re: Russian language support

Posted: Mon Jan 08, 2007 2:37 pm
by Phelan
ah.. right..
well, still, that is the same problem. It won't be forgotten, but might take a while untill we see how much work it will be.

Re: Russian language support

Posted: Tue Jan 09, 2007 12:50 pm
by virt1203
tr808 wrote:i dont think he means a total client in Russian, he simply wants the Rusian alphabet to work, so they can type with eachother in team or clanchat in Russian.
But it's will be so fine :D Plus, russian speaking GameMasters.
But of course, I've mean what we need just Unicode support in chat window ingame.
ah.. right..
well, still, that is the same problem. It won't be forgotten, but might take a while untill we see how much work it will be.
Is it realy same problem? From one side - translate client files to another language, hire russian speaking employers and from other - just inject unicode support into chat window?
Unicode - it's not only Russian language. It's all_over_the_world languages.

Re: Russian language support

Posted: Tue Jan 09, 2007 1:44 pm
by komissar
I just want to say Thank you to Phelan for answering this question.
It might not be a big deal but it shows that the new owner does indeed care for the community and it's needs.

Thank you Phelan and thank you Gameforge, keep up the positive feedback :D

Re: Russian language support

Posted: Tue Jan 09, 2007 2:09 pm
by sluggo0
virt1203 wrote:Yes, it's solve all the problems, for all world languages injecting UTF8 (unicode) to chat window.
Well, here's the only reason I see this as difficult technically. I develop business applications, since 1997 have been involved with both database based applications and 'roll your own' databases, written in ASM for multimedia broadcasting applications, and do some gamedev on the side, still would be my dream job, but I'm married with a family, so odds are low I can move into a new industry and lose salary for it.. anyhow.

MMO's are a database application through and through, for a database based app to support UTF8/Unicode character sets, the first 'port' is to convert the database, which is a large engineering project in and of itself, and has to be done very careful to avoid scrambling the data completely. That's one issue.

Secondly, MMO's are a very network intensive application. You can't run a good MMO server off a cable modem connection as you can with say a quake gameserver (arguably anyhow). UTF8 doubles the bytes transmitted for every character sent over the network. In the past, when optimizing the bandwidth of games I worked on, we optimized down to the BIT level how much traffic we sent on the wire, this means that just going and doubling the bandwidth is a very difficult decision to come to. That being said, connections are only getting faster, bandwidth only more available. And, with clever tricks to manage caches of strings you don't even send them on the network (sadly except for chat as you can't predict what a player will say, so have to send the fulll bandwidth version, unless you can come up with an compression algorithm that isn't slower than just sending it on the wire).

Thirdly, MMOs are games. Game programmers sometimes need to do 'bad' things when programming for speed's sake. This means that the impact of doubling the character size for the application is a QA nightmare. Every line of code is subject to being broken by such a change, and when working with binary data (game models, image files, sounds), the 8 bit ASCII character is used interchangably as C/C++ lack a byte type natively, and 8 bits is a byte, so we assume some things. If a byte is suddenly in some code 16 bits and some code 8 bits, weird subtle bugs are going to be everywhere. Unless the original developers were very methodical and had had experience in porting from 16bit code to 32 bit code or the like (i.e. the really old guys, who aren't that prevalent in game development anymore, old in the industry is over 30).

Depending on the forethought put into it by the very first few lines of code way back, making this change is highly non-trivial, and could require a considerable amount of time and effort to achieve.

But, it sounds as if Gameforge would at the very least consider it from Phelan's post, I just wanted to be a bit of a naysayer in terms of why it might not happen. Would be great if we all thought of all the languages of the world before writing a single line of code, sadly we all tend to be a bit biased to what language matters to US.. and historically it's been the case a lot of apps can't be ported to some languages, they'd have to be rewritten completely, which is just very expensive :(

Re: Russian language support

Posted: Tue Jan 09, 2007 3:50 pm
by virt1203
sluggo0, thank you for lection. It was very useful.
Just few comments to you programming exp. from point of IT consultant.
1. Is it really need to convert all database to UTF8? Or may be it will be enough to convert just one (two) tables for it? Is it so hard to type ALTER TABLE .. for one or two tables?
2. Don't forget what we talking about just chat window. We don't talking about all window interfaces, about NPC texts and other stuf. Just chat window.
3. I'm 100% sure what the traffic increasing which will be after implementing unicode don't influence network perfomance. Just look to other chats in other MMO.
So, count peoples who will use expanded ASCII. I think, there are drop in the ocean.

Re: Russian language support

Posted: Wed Jan 10, 2007 10:20 pm
by kaetemi
sluggo0 wrote:UTF8 doubles the bytes transmitted for every character sent over the network.
No, it doesn't. UTF-8 is kinda the same as ASCII for the first 128 bytes, then it uses that other range (128 and up), where it will use 2 characters for common characters with accents and other special common characters, and combinations of 3 characters for kanji and all that stuff. Something like that. Read the wikipedia article. Basically it means that you normally won't lose any text if some code considers it as some ASCII 8bit charset, it really only needs to be decoded to 2byte Unicode for internal use in the client (displaying the text).

You're probly thinking about UTF-16 or something.

Re: Russian language support

Posted: Thu Jan 11, 2007 8:43 pm
by sluggo0
kaetemi wrote:No, it doesn't. UTF-8 is kinda the same as ASCII for the first 128 bytes, then it uses that other range (128 and up), where it will use 2 characters for common characters with accents and other special common characters, and combinations of 3 characters for kanji and all that stuff. Something like that. Read the wikipedia article. Basically it means that you normally won't lose any text if some code considers it as some ASCII 8bit charset, it really only needs to be decoded to 2byte Unicode for internal use in the client (displaying the text).

You're probly thinking about UTF-16 or something.
Yes, I was, my bad, well that eliminates an awful lot of potential problems, it sounds kind of like a unified code page from the DOS days :) .

Re: Russian language support

Posted: Thu Jan 11, 2007 8:49 pm
by sluggo0
virt1203 wrote:sluggo0, thank you for lection. It was very useful.
Just few comments to you programming exp. from point of IT consultant.
1. Is it really need to convert all database to UTF8? Or may be it will be enough to convert just one (two) tables for it? Is it so hard to type ALTER TABLE .. for one or two tables?
2. Don't forget what we talking about just chat window. We don't talking about all window interfaces, about NPC texts and other stuf. Just chat window.
3. I'm 100% sure what the traffic increasing which will be after implementing unicode don't influence network perfomance. Just look to other chats in other MMO.
So, count peoples who will use expanded ASCII. I think, there are drop in the ocean.
Not sure, probably depends on the database environment, MySQL for example makes the database UTF-8 or not, I don't know what their DB is, if it's more 'commercial' ones like Oracle, such is likely possible, just installing Oracle took a few months to learn, much less tweaking it, which is why Oracle DBAs were so well paid at one time :P .

And regardless, with it being the chat window, it's just the tables that have lookups for the emote strings that may require tweaking.

I'm not sure if Ryzom uses a tricky way of sending these, soem recommendations I've seen involve using a lookup table at both ends, so you only have to transmit a fixed # of bytes ont he network for a phrase, once the client caches it (this lets you keep it up to date on the servers).

So, considering I was making an invalid assumptoin at the first, forgetting what UTF-8 was, it may be a lot less of an overhaul ;) . The logging would be all that would require a testing to make sure it's unaffected perhpas (assuming they log all chats on the server for a period of time).

I tend to be cautious :P

Re: Russian language support

Posted: Wed Jan 17, 2007 11:20 am
by murmadog
/bump

&#1046 ;)