Page Index Toggle Pages: [1] 2 3 ... 5
Topic Tools
Very Hot Topic (More than 25 Replies) Optimizing .lng files (Read 20,097 times)
cepheid
Senior Member
****
Offline



Posts: 516
Optimizing .lng files
Jun 27th, 2009 at 5:05am
Post Tools
cepheid wrote on May 24th, 2009 at 10:15pm:
I'll probably be submitting a patch soon for a number of (minor) updates to the English .lng files, correcting some spelling/grammar and for improved readability.Nothing ground-breaking, though. Smiley

deti, I'm working on improving the language in the English .lng files now.  They're quite long so it's taking a while. Smiley  The first patch is available now:
http://dev.3phase.com/yabb/patches/admin_lng.patch

I've noticed that there are quite a few entries that are not actually used anywhere in the code anymore.  I didn't do anything with those at this time, but IMHO they should probably be removed to improve performance (however minorly), since they're not used anywhere in the code.

Also, a question: why are some entries numbered and some given names, e.g. $admin_txt{'1'} but $clicklog_txt{'users'} ?  I assume the numbering is so that nobody has to speak English to understand, but since most development is done in English (all the YaBB comments are in English, for example), it seems like the named versions should be used instead... that would also make debugging the code a lot easier, since (at least for me) it's easier to remember what named entries might do, versus numbers which could mean anything.

Given how much overlap there is in many of the entries, naming them would also help consolidate to reduce repetition... items repeated in many language files could also be consolidated into a Shared.lng so that very common things aren't repeated.  I'll have a better idea of how much repetition is going on after I go through all the files... then, with approval, I can work on consolidating them (and, of course, updating all the .pl files which rely on those text entries).
« Last Edit: Jun 27th, 2009 at 5:05am by cepheid »  
Back to top
WWW  
IP Logged
 
Jet Li
Legacy Dev Team
Development Team
****
Offline



Posts: 6,588
Location: Hong Kong
Re: Optimizing .lng files
Reply #1 - Jun 27th, 2009 at 7:41am
Post Tools
@ cepheid

I will put later today to CVN/SVN after reviewed. Smiley And thnx for correction. Smiley
  

PM me for YaBB Installation Service
Back to top
WWWGTalkFacebook  
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Optimizing .lng files
Reply #2 - Jun 27th, 2009 at 7:47am
Post Tools
Jet Li wrote on Jun 27th, 2009 at 7:41am:
And thnx for correction.

My pleasure.  There will be more coming.  If anyone knows the answer to the question above, though, it'd be cool to know.
  
Back to top
WWW  
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Optimizing .lng files
Reply #3 - Jun 27th, 2009 at 2:34pm
Post Tools
Thanks cepheid, JetLi will do it.

To your question: I don't know. I assume that numbers are easier, using names you must think about how to call it. Smiley
I also think that it is easier to know for what each variable is by the name of it and not by the number, but there are really many to change!!!

Some variables may have the same text in different language files, be aware that not every .lng-file is loaded everywhere!

Also think on the following if you start changing things: Other languages sometimes have other sentence structure, so we sometimes need two variables (one before and one after another entry) where we need only one in English.
  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Optimizing .lng files
Reply #4 - Jun 27th, 2009 at 2:41pm
Post Tools
deti wrote on Jun 27th, 2009 at 2:34pm:
I also think that it is easier to know for what each variable is by the name of it and not by the number, but there are really many to change!!!

Yeah, there are certainly a LOT!  Maybe one day I'll try to change it... though that would be a major undertaking.

deti wrote on Jun 27th, 2009 at 2:34pm:
Some variables may have the same text in different language files, be aware that not every .lng-file is loaded everywhere!

Oh, I know.  But, if there are a lot of shared items, then the Shared.lng could be loaded everywhere.  I don't know if that would really help or not.  Within a given .lng file, though, there's certainly no need for duplicates... even if we can't remove duplicates between files, we should be able to do it within files. Smiley

deti wrote on Jun 27th, 2009 at 2:34pm:
Other languages sometimes have other sentence structure, so we sometimes need two variables (one before and one after another entry) where we need only one in English.

Yeah, I was wondering about that.  But if you only need one in English, the second one would be blank, right?  But, there are no blank ones... so I don't think this will be a problem.  In any case, I will be very careful if/when I do prune anything... items that really are truly identical (there are a few!) can probably be combined, and items that are truly not used (there are more than a few!) can probably be removed, because they're not used anywhere in the code.  But don't worry, I will be very careful!!

My first focus will be to clean up the language; I'll worry about removing/combining later, since that's the tougher and also less necessary part.
  
Back to top
WWW  
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Optimizing .lng files
Reply #5 - Jun 27th, 2009 at 2:59pm
Post Tools
cepheid wrote on Jun 27th, 2009 at 2:41pm:
Within a given .lng file, though, there's certainly no need for duplicates... even if we can't remove duplicates between files, we should be able to do it within files.

Absolutely!!

cepheid wrote on Jun 27th, 2009 at 2:41pm:
But if you only need one in English, the second one would be blank, right?

Right.

cepheid wrote on Jun 27th, 2009 at 2:41pm:
items that really are truly identical (there are a few!) can probably be combined

I think so.
cepheid wrote on Jun 27th, 2009 at 2:41pm:
... and items that are truly not used (there are more than a few!) can probably be removed, because they're not used anywhere in the code.

Yes, this would be a really good cleanup!!!
  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
OH Eng
Past Team Members
Documentation Team
Offline



Posts: 4,026
Location: Pensacola, Florida USA
Re: Optimizing .lng files
Reply #6 - Jun 27th, 2009 at 3:22pm
Post Tools
Quote:
Also, a question: why are some entries numbered and some given names, e.g. $admin_txt{'1'} but $clicklog_txt{'users'} ?  I assume the numbering is so that nobody has to speak English to understand, but since most development is done in English (all the YaBB comments are in English, for example), it seems like the named versions should be used instead... that would also make debugging the code a lot easier, since (at least for me) it's easier to remember what named entries might do, versus numbers which could mean anything.


Over time, as features were added that required use of data in a lng file, they were sometimes added as new entries when others already existed that would have worked.  Sometimes an existing mod was added and that mod - if installed as a mod - added to the lng file rather than use what's there.  Why?  I don't know... maybe the mod writer didn't spend the time to look through the files for the words used, and that resulted in duplications.  When added as a feature, the .lng references were added also, and the code was not always scrubbed to alter every reference to the mod-added .lng data, changing it to existing .lng data that would have worked.
Point is, they were not necessary, but the goal was to get the mod added and working, the when testing it was only tested to see it it worked or didn't.

This is not to justify the duplicates, certainly it would be better if none existed, but you will find this kind of thing all over the code.  You can look in the Bugs Fixed archive and see many instances where deti has removed a lot of duplicate things.

Part of that too happens when you have one person developing one part of the code and another developing something different.  Dev A adds a bit to the .lng not knowing Dev B is going to add the same thing.  It's not intentional, it just happens.  This, plus the adding thing also help explain why different conventions (numbers vs. names) are used in different spots.

One thing to keep in mind, and I'm sure you have, is people often customize these lng files.. greeting messages, error messages, etc. and that can cause problems upgrading version to version if you change the .lng source of the information.  But there is probably no better time to do it than when making a new version as in Y3. 

Food for thought.  Something that might be handy to have is some kind of cross-reference of changes where one can look up the old value and see what is taking its place.
For example, if $admin_txt{'1'} and $admin_txt{'87'} are the same, and you change the reference in the code from 87 to 1, deleting the duplicate 87, a user who altered 87 might like to know where they now have to make the modification they need. 

« Last Edit: Jun 27th, 2009 at 3:33pm by OH Eng »  

 
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Optimizing .lng files
Reply #7 - Jun 27th, 2009 at 3:49pm
Post Tools
OH Eng wrote on Jun 27th, 2009 at 3:22pm:
you will find this kind of thing all over the code.

Indeed, I have... there are lots of places in the code (not just the .lng files) where some things happen redundantly.  Cleaning that up is actually one of my future projects, as well... I plan to go through every file and every line, seeing where I might be able to do local optimization (removing redundancy or reducing code).  Global optimization (reducing and/or rewriting entire functions) is more difficult and generally a better payoff, but we should be able to get some good mileage out of local optimization alone.

OH Eng wrote on Jun 27th, 2009 at 3:22pm:
For example, if $admin_txt{'1'} and $admin_txt{'87'} are the same, and you change the reference in the code from 87 to 1, deleting the duplicate 87, a user who altered 87 might like to know where they now have to make the modification they need.

Excellent point.  I would probably include comments in the code where any line was removed, pointing to the appropriate entry that is used instead.  That should take care of the user doc while not impacting performance.  Eventually, the comments can be moved out into a separate "deprecated map" file.

Thanks for the helpful comments! Cool  Hopefully I can make actual headway on this project - there are so many .lng files, and so many entries in each one, that this will be a long endeavor... and I'm only doing the English ones! Shocked
« Last Edit: Jun 27th, 2009 at 3:49pm by cepheid »  
Back to top
WWW  
IP Logged
 
batchman
Support Team
****
Offline



Posts: 376
Location: Orlando, FL
Re: Optimizing .lng files
Reply #8 - Jun 27th, 2009 at 4:03pm
Post Tools
I make no promises, but I was an English major (as far as the spelling and grammar go), and can probably spot duplicate entries.

If desired, tell me to work on one particular language file, and I'll go through and see what I can do. If it works well, send more my way.

An extra way I might be able to help with one of my favorite pieces of software.
  
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Optimizing .lng files
Reply #9 - Jun 27th, 2009 at 4:12pm
Post Tools
batchman wrote on Jun 27th, 2009 at 4:03pm:
If desired, tell me to work on one particular language file, and I'll go through and see what I can do. If it works well, send more my way.

I just did Admin.lng... if you are up for it, try Backup.lng.  Do you know how to do a unified diff?

For now, I wouldn't actually remove any duplicates - just note them.  Also, be careful when editing stuff for grammar because a LOT of the entries are partial phrases, and are that way deliberately, so they seem like improper grammar even when they aren't.  It can be confusing...
  
Back to top
WWW  
IP Logged
 
batchman
Support Team
****
Offline



Posts: 376
Location: Orlando, FL
Re: Optimizing .lng files
Reply #10 - Jun 27th, 2009 at 4:20pm
Post Tools
I have no idea what a unified diff is ... but I can look for duplicates and mis-spellings and the like.

(Backup is probaby a nice one to start on ... one of the newer features, less work done on it over the years, probably already much leaner than many of the others.)

Edited:
Oops ... it's the newest version we're working on, these days ... can anybody tell me where I can find the files for download?
« Last Edit: Jun 27th, 2009 at 4:24pm by batchman »  
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Optimizing .lng files
Reply #11 - Jun 27th, 2009 at 4:28pm
Post Tools
batchman wrote on Jun 27th, 2009 at 4:20pm:
I have no idea what a unified diff is ... but I can look for duplicates and mis-spellings and the like.

OK, just submit the new file when you're done editing it. Smiley

batchman wrote on Jun 27th, 2009 at 4:20pm:
can anybody tell me where I can find the files for download?

The SVN repo: http://yabb.svn.sourceforge.net/viewvc/yabb/
  
Back to top
WWW  
IP Logged
 
batchman
Support Team
****
Offline



Posts: 376
Location: Orlando, FL
Re: Optimizing .lng files
Reply #12 - Jun 27th, 2009 at 6:02pm
Post Tools
Actually going over one of these that is a quite small language file just shows me how much work some of the larger ones are going to be.

I have gone through this file and fixed a small number of misspellings and grammar errors. I have found there is one line repeated twice. The text for '21' and '25' are the exact same.

For easily figuring out what I have done, I have added a couple of lines explaining what I did / found at the very beginning, that can be stripped before replacing the file.

Most changes were very small (than, instead of then or successful rather than successfull), a few involved slightly re-wording to make the text flow more smoothly.

For trying to figure out which lines of code are no longer referenced anywhere, how would you figure out which files to check? Or would you check all the /Sources, /Admin, and /YaBB2 files for every line?
« Last Edit: Jun 27th, 2009 at 6:05pm by batchman »  

JB_Backup.txt (Attachment deleted)
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Optimizing .lng files
Reply #13 - Jun 27th, 2009 at 6:12pm
Post Tools
batchman wrote on Jun 27th, 2009 at 6:02pm:
Or would you check all the /Sources, /Admin, and /YaBB2 files for every line?

Most of the lines, I recognize... the ones I didn't recognize were often the ones that weren't used.  I just grep'd for those to confirm they weren't used.

Yes, Admin.lng took me a few HOURS to go through...

There are still a few small things in Backup.lng that I can see to fix, but not major items.  Let me tackle a few other files and we'll see how to split the workload. Smiley
  
Back to top
WWW  
IP Logged
 
batchman
Support Team
****
Offline



Posts: 376
Location: Orlando, FL
Re: Optimizing .lng files
Reply #14 - Jun 27th, 2009 at 6:34pm
Post Tools
'27' is not used in the Backup.pl anywhere.

So what did I miss, and does it fall under the header of not knowing the technology portion well enough to correct it?
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 3 ... 5
Topic Tools
 
  « Board Index ‹ Board  ^Top