And here's the converter. I have a Y2 output driver and an Eblah input driver. I chose Eblah because it was flatfile and already documented.

Try it out, and tell me what you think. You need to put the Prefs, Messages, Boards, and Members directories from Eblah into the convert directory. Chmod everything 777 or as you did for YaBB.

I'll post how to make your own input driver later.

Oh, and ignore the "NewFile"s. They serve no purpose except to include those directories.

Ok, I'll answer the easy question first. The problem with having too much data doesn't really happen here. The user of the converter is allowed to set a maximum amount of time for each step to run. If it exceeds that time, the converter jumps out of the conversion loop and dumps anything that needs to be saved to a Perl file. When the user clicks "next", it's loaded into memory again. It doesn't auto-reload, but it should be perfect at saving and resuming, as long as it's within the same time period.

Now, for the howto. Your existing code will need to be separated and transfered into 3 different types of routines for this to work. I'll explain the basic format of the indriver here, as well as explain the Init routine.

This might seem really complicated. It probably is, compared to most of the programming I do. But it's fairly easy to understand if you understand the following points:
- There's 3 main parts of the file: The setup part, which determines what parts of the conversion you can do and what code is executed when; the initial/finish routines, which are generally simple and similar to other routines of the same type; and the conversion routines.
- The save/resume work in progress is very generic. For saving, you just need a list of what variables to store and what they're named. For saving, you just call the resume routine and it does the rest.
- Use localized variables and filehandles all the time. This works wonders to prevent unexpected results of variables that weren't intended to be global.

I recommend taking my Eblah input driver (or if you want a fictional board format, I can provide that input driver) to start with and using the basic framework, especially the Init/Finish routines. If you know what you are doing, you can write a second converter in less than 10 hours, as I did with the Eblah one.

For the input driver, here's the basic form:

package indriver;

# strict + warnings is good
use strict;
use warnings;

our %CONFIG = (
'BOARD_TYPE', 'the board type it converts (e.g. phpBB 2)',
'NAME', 'name of the converter',
'VERSION', 'Alpha'
);

# What I can convert
our @CONVERT = qw(Boards Categories Messages Members);
our %SUBROUTINES = (
'Boards', 'BoardConvert',
'Categories', 'CatConvert',
'Messages', 'MessageConvert',
'Members', 'MemberConvert'
);
our %INITSUBROUTINES = (
'Boards', 'BoardConvertInit',
'Categories', 'CatConvertInit',
'Messages', 'MessageConvertInit',
'Members', 'MemberConvertInit'
);
our %FINISHSUBROUTINES = (
'Messages', 'MessageConvertFinish'
);
 

Using warnings and strict is optional, but recommended.



The %CONFIG hash does nothing at the moment, but it's good to have in there for when it might be used.

Now, the @CONVERT array is important. It specifies what items the converter can convert. So far, there's four that are supported by YaBB's output driver: Boards, Categories, Messages, and Members. The converter uses the @CONVERT arrays of the input and output driver to find out what to convert.

Each item in the @CONVERT array is matched up to a subroutine using the %SUBROUTINES, %INITSUBROUTINES, and %FINISHSUBROUTINES hashes. You can call the subroutines that match up anything you want.

We'll start with the easy subroutines, which are the init/finish ones.

The input driver's init routine for each step has a very important step: telling how many steps to run the loop for. It also sets up package globals needed throughout the conversion process, like matching what board goes to what category, or what topics are sticky.

Here's the BoardConvertInit routine from my Eblah indriver

sub BoardConvertInit {
	my $fh; # Use a localized variable for a filehandle to prevent problems.

	open($fh, "<$convertdir/Boards/bdindex.db") || die "Can't open $convertdir/Boards/bdindex.db: $!";
	@boardlist = <$fh>;
	close($fh);
	$maxsteps = scalar @boardlist;

	open($fh, "<$convertdir/Boards/bdscats.db") || die "Can't open $convertdir/Boards/bdscat.db: $!";
	@catlist = <$fh>;
	close($fh);

	return 1;
} 

The first interesting thing is the $fh variable. It's used exactly the same as a filehandle. I consistently use $fh when I need an anonymous filehandle.

Second, I use "die". Die outputs HTML, but you don't need to worry about the templating area, which is handled in the converter. Same goes for "warn", which is also considered a fatal error.

I open up the board index file and count up the boards present. Since there's one board per line, I capture the whole list in the global @boardlist. I use it in forced scalar context to count up the maxsteps.

Something important is that maxsteps is actually the number of steps, plus one! It works this way due to how I made the for loop. It usually works out perfectly since you often count the lines in the file.

A change from the Beta 1.0 and Beta 1.1 versions is that you should return the number of steps. All you have to do is return it instead of store it in $maxsteps.



If you run out of time, you can resume work in progress like this:

	if($_[0] eq 'resumeworkinprogress') {&ResumeWorkInProgess();}
	else {
	# The rest of the init code goes here
	}
 

That's it. You just need to make sure you're saving work in the corresponding Finish routine. You'll notice that sometimes I don't save/resume in the input driver. That's due to the fact that I only need to open 1 or 2 files, and there's no real problem with counting it again.

---

I'm running out of characters in this post, so I'll continue typing in my next post.

( oh, that was a scary 12 hours !!!!)

Yes - can see what you're doing now. Eblah looks safely the easiest to pick as the baseline.

Any thoughts on :

Where the

use DBI ;   

is going to be best placed for the db driven ones?

I've had to add an extra screen in on my code, to allow for confirming that PERL can see and access the input database, since your kind of #stuck# otherwise, and then (this is Webboard for you   confirm which (board) to do the convert on.

An extra hash for the database connection details perhaps?

our %DBASE= (
'DBASE_TYPE', 'mysql/oracle/sql server/access',
'DB_NAME', 'name of the database',
'SERVER_NAME', 'name of the database server (probably localhost, but you never know),
'LOGIN', 'sa login id',
'SAPWD', 'sa password'
'VERSION', 'Alpha'
); 

I'll see if I can add options to the screen right before it converts, where you input the maximum time and see what can be converted. Expect this in a week or so -- school's being evil to me now).

The "use DBI;" should be placed right by the "use warnings", I think. It'll error out before any conversion is attempted this way.

As for the generic time conversion routine, there's a slight problem. The converter doesn't know what type of time to expect. You can do like I did with the Eblah one if you have to, which is adding a string-to-time routine. I'd perfer the input and output drivers to be fairly self-contained. You'll need to best figure out a way to convert into the Unix time() format.

I'll write up how to handle the Message Conversion step later.

Hm. I guess something like that might be worth having. I'll think about it for Beta 2.

Looks like you're doing good so far. I'll see about a Beta 2 with the changes I mentioned soon.

Sure.
PLease bear in mind the version I have is currently wrapped around a variation on setup.pl, so the forum-specific parts are not as yet separated.

This one is complete up to 'members', and I'm working through the business of mapping the 'messages' over.

I am trying out the converter, but can't get step 4 to work.

The following message is displayed:
Error! Use of uninitialized value in split at <root>/cgi_bin/converter/Output/YaBB2.pl line 236.

Unfortunately I am not experienced in Perl scripting.
It has something to do with the Prefs/Ranks2.txt file, but I can't figure out the relationship between that file and the user data files.

Any pointers?

By the way, YaBB works great!