Page Index Toggle Pages: [1] 2 
Topic Tools
Hot Topic (More than 10 Replies) Quick-Guide Upgrade procedure incorrect: data loss (Read 10,415 times)
cepheid
Senior Member
****
Offline



Posts: 516
Quick-Guide Upgrade procedure incorrect: data loss
May 24th, 2009 at 10:23am
Post Tools
In trying to upgrade a YaBB 2.1 forum to 2.4, I noticed that the upgrade procedures listed in the Quick-Guide are incorrect, in that the way the code is implemented, one of the steps prevents the old settings from being properly migrated to the upgraded forum.

Specifically, for upgrading from v2.1, the Quick-Guide says:
1) Install a brand new v2.4
2) Run Setup
3) Set the new forum to Maintenance Mode
4) Copy various files (Boards, Members, Messages, Variables) from the old board to the new board.
5) Log in, go to Settings, and save it.
6) Rebuild notifications, membership, message list, etc.
7) Check all your settings and modify as desired.

Step 3 is what causes the problem... after running step 3, copying the settings from the old board doesn't matter because they'll be ignored.  If you swap steps 3 and 4 (do step 4 first), or if you skip step 3 entirely, it all works fine.

The reason for this is that setting Maintenance Mode (step 3) rewrites the new Settings.pl and sets the $settings_file_version = "YaBB 2.4" ... when that happens, the upgrade procedure isn't run anymore!  So, if the old files are copied (step 4) after setting Maintenance Mode (step 3), none of the old settings files will be read because the $settings_file_version has already been modified and the upgrade procedure gets skipped.

If you do step 4 first, the old settings files are in place before the version number is changed - the upgrade procedure gets run properly and all is well.

So... to make a long bug report short, the fix for this is EITHER:
1) Rewrite the Quick Guide to swap steps 3 and 4 (copy the old files BEFORE setting Maintenance Mode), or
2) Change Admin/NewSettings.pl so that setting Maintenance Mode does not change anything in the Settings.pl file except for the maintenance_mode flag... in particular, ensure that $settings_file_version does not change.

#1 is a quick fix, but in principle not the right move because you want your new forum in Maintenance Mode before copying all those files over, to ensure that nobody can log in, make posts, etc. while you're halfway through the upgrade.

Thus, #2 is the more appropriate solution, because it ensures that the forum is in Maintenance Mode during the upgrade, and also it's correct "philosophically" since setting Maintenance Mode should do only that - set the maintenance flag, without modifying anything else.

(I can't tell you how many times I deleted and reinstalled my board before figuring this bug out.... it was really frustrating.)

OK, I've discovered more data loss:

My ban information (from Variables/ban.txt) gets lost even if I follow the procedure above, which otherwise works fine for everything else.

I don't know why this happens, because the upgrade procedure (in &SaveSettingsTo within Admin/NewSettings.pl) clearly does read Variables/ban.txt and appears to parse it just fine... but for whatever reason, that information never gets saved into the Settings.pl file.  The code looks fine on a cursory inspection, but since the end result is not the correct one, there's a bug somewhere in there.

Even worse, Variables/ban.txt gets deleted after the upgrade so even though the code to save the banned IPs/emails/usernames doesn't work, the code to delete the file does and the ban information is lost "forever" that way.  (Thankfully, I have a backup.)

So, the code in &SaveSettingsTo which reads and parses ban.txt should be examined, since there's data loss in there and I have no idea why.  (I'll try to step through the code more thoroughly when I get a chance, but I wanted to report this ASAP.)
« Last Edit: May 24th, 2009 at 12:00pm by Jet Li »  
Back to top
WWW  
IP Logged
 
OH Eng
Past Team Members
Documentation Team
Offline



Posts: 4,026
Location: Pensacola, Florida USA
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #1 - May 24th, 2009 at 2:03pm
Post Tools
That is an excellent catch, cepheid!  We have had a number of users report data loss on upgrade, not always the same data, but some.

But here is a questions for you... I tested this procedure with two different Y2.1 boards, upgrading them to 2.4, and all the settings made it just fine.  I opened both Settings.pl files (after upgrade) and all the data was there.  So how did that happen if MaintMode kills the settings transfer procedure?  Undecided
  

 
Back to top
 
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #2 - May 24th, 2009 at 2:55pm
Post Tools
Thanks cepheid for this catch.

To problem1: What do you think if we put the forum in maintenance mode from beginning on? So that it is in Maintenance mode right after setup. That way we don't need to put it in maintenance mode and also it's more safe if the forum in in maintenance mode right after install.

To problem 2: I will give a look.
Edited:
In NewSettings.pl we have:
Code
Select All
	my ($type, $bannedlist) = split(/\|/, $line, 2); 


must be
Code
Select All
	my ($type, $bannedlist) = split(/\|/, $_, 2); 


Please test it cepheid. Thanks!

New NewSettings.pl in CVS.



OH Eng wrote on May 24th, 2009 at 2:03pm:
I opened both Settings.pl files (after upgrade) and all the data was there.

Are you absolutely sure your new board was in maintenance mode?
« Last Edit: May 28th, 2009 at 7:38pm by deti »  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
OH Eng
Past Team Members
Documentation Team
Offline



Posts: 4,026
Location: Pensacola, Florida USA
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #3 - May 24th, 2009 at 7:42pm
Post Tools
Yes I am 100% certain.  You might recall you asked me to test the upgrade procedure and I did it on an existing 2.1 test forum and I also had a backup of another 2.1 forum I no longer use (upgraded it to 2.2 long ago).  I reinstalled it and upgraded it also.  I followed the procedures exactly, step-by-step to make sure they worked, and both forums upgraded without any problems.

So I don't know how that could have happened and all my settings moved into the new Settings.pl if this prevents it.  Was going to send it to you via PM but the file is too big to include in a PM.
  

 
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #4 - May 24th, 2009 at 9:32pm
Post Tools
deti wrote on May 24th, 2009 at 2:55pm:
To problem1: What do you think if we put the forum in maintenance mode from beginning on?
That's totally fine, for sure, as long as you write the initial settings with $maintenance=1 rather than using the "set maintenance mode" function, since that just automates what happens manually. Smiley

Of course, the Quick-Guide would have to be rewritten either way.  You should actually ensure that any new forum (upgrade or not!) starts out in Maintenance Mode, because it forces the admin to go through the Settings at least once - that's a good thing.

deti wrote on May 24th, 2009 at 2:55pm:
Please test it cepheid. Thanks!

I'll try to test it when I have a chance, but I've upgraded the two production forums already so it'll have to be a dedicated test later on... I spent 5 hours on this last night so I'm a little spent. Smiley  I'll do it ASAP.

OH, not sure why it worked for you, but both problems were entirely reproducible multiple times for me when I followed the steps in the Quick-Guide explicitly.  I ran it through about 5-6 times while looking for the problem...
« Last Edit: May 24th, 2009 at 9:34pm by cepheid »  
Back to top
WWW  
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #5 - May 24th, 2009 at 10:07pm
Post Tools
New
Setup.pl

upgradeY2.html

in CVS.

I changed the text in the Q-G from:
Quote:
Log in to your NEW forum (remember, your default username is "admin" and password is "admin" too) and place it in maintenance mode: "AdminCenter" => "Maintenance Settings" => "Maintenance Mode?"

to
Quote:
Log in to your NEW forum (remember, your default username is "admin" and password is "admin" too), let it in Maintenance Mode as it is after Setup.


I hope it is OK for you guys, otherwise tell me a better text.

@ OH Eng
The only thing I can imagine is, that your old forum did not have settings in that files that are converted to the new format and therefore you did not notice that they were missing. But definitely putting it into maintenance before you move the files from the old to the new, disable the ability to convert this files and delete them afterwards. This should be fixed now.


I consider this bug(s) fixed now.
« Last Edit: May 24th, 2009 at 10:08pm by deti »  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #6 - May 24th, 2009 at 10:15pm
Post Tools
deti, the new text looks fine though you might want to say "leave" instead of "let."  Otherwise, it looks good. Smiley 

Actually, though, there's no reason to even log in to the new forum at all except to see that it's there... after Setup.pl is complete, one can move straight to copying all the files, then log in as your old administrator self.  So, you could even just replace that entire line with, "Your NEW forum is already in Maintenance Mode; leave it that way while you complete the following steps."

Either way is fine, of course.

I'll probably be submitting a patch soon for a number of (minor) updates to the English .lng files, correcting some spelling/grammar and for improved readability.  Nothing ground-breaking, though. Smiley
  
Back to top
WWW  
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #7 - May 24th, 2009 at 10:30pm
Post Tools
cepheid wrote on May 24th, 2009 at 10:15pm:
"Your NEW forum is already in Maintenance Mode; leave it that way while you complete the following steps."

Good idea!

New
upgradeY2.html
in CVS.

cepheid wrote on May 24th, 2009 at 10:15pm:
I'll probably be submitting a patch soon for a number of (minor) updates to the English .lng files, correcting some spelling/grammar and for improved readability.

Constructive help is always appreciated! Smiley
  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
deti
Legacy Dev Team
Development Team
****
Offline



Posts: 2,650
Location: Prien am Chiemsee, Germany
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #8 - May 25th, 2009 at 10:05pm
Post Tools
Off topic here but on the other side in topic too:

New
/Sources/Decoder.pl
/Sources/Search.pl
in CVS.

Validation Code fixed, told and repaired by cepheid.
  

Was immer Du tun kannst
oder erträumst tun zu können,
beginne es.
Kühnheit besitzt Genie,
Macht und magische Kraft.
Beginne es jetzt.
Whatever you can do
or dream you can,
begin it.
Boldness has genius,
power and magic in it.
Begin it now.
J. W. Goethe
Back to top
WWW  
IP Logged
 
Corey Chapman
YaBB Administrator
*****
Offline



Posts: 10,024
Location: Rock Hill, South Carolina

None
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #9 - May 27th, 2009 at 5:31pm
Post Tools
Because this issue deals with upgrades and installations rather than the actual forum functionality, I think we can make these few changes to the existing 2.4 package and re-release it to the SourceForge repository.  The only downside is resetting the download count on their site for this release.  I'd rather not take a chance of us being responsible for data loss.

Those who have already downloaded won't be missing out on anything because this is just regarding installation which they have already done.

Send me the updated original 2.4 packages (.zip and .tar.gz) with the changed guide and settings script.  Once I upload them to SF, I'll update the description on the downloads page and at SourceForge to identify that a change to the package was made.
  

Back to top
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #10 - May 27th, 2009 at 5:53pm
Post Tools
Corey, you don't want to also include bug-fix updates, especially the validation code issue?
  
Back to top
WWW  
IP Logged
 
Corey Chapman
YaBB Administrator
*****
Offline



Posts: 10,024
Location: Rock Hill, South Carolina

None
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #11 - May 27th, 2009 at 6:21pm
Post Tools
No.  Other changes such as the validation code issue cannot be included in this.

The reason I'm even allowing it is that it is not a forum functionality fix or change, it's regarding installation.  Those who already installed will not miss out on anything.

We can't release fixes or improvements in an existing release or it will cause confusion.  Nobody in the software world does this.  We have to change the version number so everyone is aware that updates have been made, and we must ensure that the changes have been properly tested.  The only time we've ever violated this rule is within a couple days after release because of a last minute mistake or bug found that was serious.
  

Back to top
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #12 - Jun 6th, 2009 at 2:43am
Post Tools
Corey Chapman wrote on May 27th, 2009 at 5:31pm:
Because this issue deals with upgrades and installations rather than the actual forum functionality, I think we can make these few changes to the existing 2.4 package and re-release it to the SourceForge repository.

Corey, just wondering if this was done yet?  In the Upgrade forum, there was a user asking about upgrading a couple of days ago... OH Eng provided the workaround detailed in the OP, so I'm guessing the updated Setup.pl and Quick-Guide are not yet in the distribution.

Just curious; not trying to press or anything.
  
Back to top
WWW  
IP Logged
 
djblamire
Full Member
***
Offline



Posts: 157

None
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #13 - Jun 7th, 2009 at 9:58am
Post Tools
I'm planning on doing the upgrade shortly, and wondering whether the updated install is now on sourceforge ? - As 'OH Eng' is still discussing the workaround on another thread.

Thanks in advance,

Daniel
  
Back to top
 
IP Logged
 
cepheid
Senior Member
****
Offline



Posts: 516
Re: Quick-Guide Upgrade procedure incorrect: data loss
Reply #14 - Jun 7th, 2009 at 10:26am
Post Tools
djblamire wrote on Jun 7th, 2009 at 9:58am:
I'm planning on doing the upgrade shortly, and wondering whether the updated install is now on sourceforge

I don't think it's in the official release package yet.  You may have to download the CVS version if you want the fixes discussed here, until/unless Corey says otherwise...
  
Back to top
WWW  
IP Logged
 
Page Index Toggle Pages: [1] 2 
Topic Tools
 
  « Board Index ‹ Board  ^Top