Reference For Extending The CVW Updater

The update process depends on both the updater program (and it’s component files) and the CVW database. The actual object data is provided by the database. The updater facilitates the process and provides a user interface.

How is the data gathered?

In the database, each type of object has a verb named recore_lines. This verb writes out all the important data specific to that object type. For instance, a note object would need to write out its text property, a URL would need to write out its path, and so on. The data is written out in the form of moo code, which can later be read in to recreate the object in another CVW.

There is a $tree object on the server (#94) that coordinates and writes out the data. It has a verb named recore_send that gathers the recore_lines data from every object in the CVW, orders that data, and writes it out. The order of this data is very important for several reasons.

Users must be recreated first. This is because, as the other objects are recreated, their ownership (and other properties) must be set to a valid user, thus the users must preexist the objects. Exits must be recreated last because they link one room to another, thus the rooms must preexist the exits. The creation of other objects is unordered.

After creation of all objects, some data must be written out to ensure the objects behave as before. There are three instances of this: 1) all shortcuts must be subclassed off their parent and designated as shortcuts, 2) all users must have their features reapplied, and 3) all groups must have their shared_objects (i.e. objects they own) property set. The shortcuts issue must be resolved after object creation, to ensure that the parent exists and to ensure that data on the shortcuts won’t be overwritten by the creation of other objects, thereby breaking their inheritance. The users’ features must be added after object creation (rather than at the time the users are created) because all the feature objects must exist. And the groups must have the ownership reset, because while objects were being recreated (and setting their ownership property) the group may not have existed yet, thus it would not have recorded the information. These three tasks can be done in any order with respect to each other, but they must all occur after object creation is complete.

Lastly, the recore_send verb writes out the system settings. This does not have to be done last. It can be done at any point in the process. I just chose to configure the system information after all objects have been established and the CVW is in working order again.

How is the data written out?

The $tree object has a verb named tell_to_client. This verb takes a string and writes it out to the CVW server’s log file. It flags the line by prepending it with the string: "RECORE: ". So the data is actually written from the database to the server’s log file.

What is the Primer file?

The primer file contains moo code that prepares the database to write itself out. If there are any changes that must be made to the old database in order to properly write it out, those changes are made here. For instance, sometimes it is necessary to change the recore_send verb to incorporate data that was previously not saved during an update, or to alter the format in which the data is written.

The primer file must always end with the following code:

@verb #0:server_started this none this
@program #0:server_started
player = #2;
$tree:recore_send();
delete_verb(#0, "server_started");
shutdown("Shutting down after recore send.");
.
;shutdown("Finished priming")

What is the Final-Recore file?

The final-recore file contains moo code that ties up any loose ends after the update. If there are any changes required to make the new database complete or usable, those changes are made here. For instance, if a new property were added to cache data that had previously been calculated on demand, then the final-recore file would contain code to calculate and initialize all the values.

The final-recore file must always end with the following code:

@quota #2 is 1000
@verb #2:_fix this none none
@program #2:_fix
this.home = $player_start;
move($helpers,$help_room);
.
_fix #2
@rmverb #2:_fix
;shutdown("Finished loading user objects");

What does the updater do?

The updater walks the user through each of the steps necessary to write out the old database, and read that information into the new one. There are four steps to this process:

Step 1: Prime the old database.

In this step, the primer file is read into the old database, modifying the database to write itself out.

The updater begins by launching the old database on port 1111 of current machine. It then reads the name of the database file and the admin’s password entered by the user. The updater creates a file named "primehead" where it stores the connection string ("connect admin " followed by the admin’s password) and a string denoting the accepted database versions. This version string is hard-coded into the updater binary. It is defined by the variable versionToUpgrade at the start of the updater.tcl file.

Next, the updater calls the cvwconnect binary passing it the contents of the primer file. Cvwconnect reads the connection string from the first line of the primehead file, and uses that to connect to the server. Cvwconnect then reads and parses the version string from the next line of the primehead file, and uses that to verify that the database is the right version to upgrade. If either of these checks fail (cvwconnect cannot log in, or the database is the wrong version), cvwconnect will abort and create a file named "cvwconnect_error" containing the error code. If both checks succeed, cvwconnect will input the data passed in (the contents of the primer file) and exit, deleting any cvwconnect_error file that may exist.

After cvwconnect exits, the updater checks for the existence of the "cvwconnect_error" file. If this file exists, the updater reports the given error and aborts step one. If it doesn’t exist, the updater waits for the database to shutdown and reports the successful completion of step 1. At this point there will be a new database called "primed.db" that is ready to write itself out in step 2. The time required for step 1 will depend upon the size of the primer file, but, given that the file is generally less than 100 KB, this step should take only a couple minutes. The primer file would have to be several MB before there would be any noticeable delay.

Step 2: Write out the database.

In this step, the server writes the object data to its log file and then shuts itself down.

The updater simply launches the database created by step 1, the "primed.db", and redirects the output to a "recore.log" file. The database was modified in step 1 to write out its data upon launch and then shutdown when finished. The updater waits for the server to shutdown and then reports the completion of step 2. While waiting, the updater retrieves file size and status info about the recore.log file and reports it, so the user can view the progress.

If an error occurs anywhere in the server, it will abort the process of writing itself out, and will never reach the final stage to shut itself down. The updater has no way of knowing that an error occurred and will simply wait forever in this step. If after several minutes, the file size and modification date of the recore.log file have not changed, an error probably occurred.

Do not be alarmed if this step takes a long time. Depending on the size of the database, this step could take anywhere from a few minutes to several hours.

Step 3: Parse the log file.

In this step the updater parses the recore.log file, performs clean up, and saves the result as "recore.in".

The parsing process consists of stripping any lines that don’t contain the RECORE flag. For those lines that do contain the flag, the generic log information, such as date and time, and the flag itself are stripped, leaving only the actual moo code. Finally certain lines, specifically those beginning with "@quota" and those that set creation and modification properties, are relocated to the end of the file. This is because the act of recreating the objects will modify these values, so that they need to be set after everything else is completed.

After parsing the recore.log file and creating the recore.in file, the updater will append the contents of the "final-recore" to the recore.in file. The recore.in file will then be ready for step 4.

The duration of this step will depend on the size of the recore.log file and can range anywhere from a few seconds to several minutes (or perhaps even an hour for extreme cases). The size of the recore.log file is based on both the size of the original database and the type of objects it contains (e.g. notes have more data to write out than URLs, etc). Generally though, the recore.log file is roughly the same size as the original database.

Step 4: Create new database.

In this step, the recore.in file is read into the old database, recreating the old objects and performing any clean up or initialization required.

The updater begins by launching the core database on port 1111 of current machine. It then calls the cvwconnect binary and passes it the contents of the recore.in file. This time cvwconnect is called with arguments telling it not to check for errors, since the core database is provided with the updater and its version and admin password are known. Cvwconnect will input the data passed in (the contents of the recore.in file) and exit. This will recreate all the users and other objects and restore the system settings, resulting in a replica of the old database with all the enhancements of the new core.

After cvwconnect exits, the updater waits for the database to shutdown and then reports the successful completion of step 4. After this step, there will be a new database called "new.db" in the updater directory. That is the updated database ready for use.

How does the Updater change from version to version?

Each updater will have a unique primer file and final-recore file. These files are customized for each upgrade to make any necessary database changes.

The actual code in "updater.tcl" rarely changes. However, at the start of the file are a number of variables that tailor the updater to a specific version. There is a titleMsg, a newCoreDatabse, an oldServer, a newServer, and a versionToUpgrade. There is also a primerFile, a finalrecore, an errorfile, and an outputDelimiter, but they haven’t changed since inception. All of these variables are strings.

The titleMsg is a string the updater simply displays for users to distinguish the updater’s version.

The newCoreDatabase is the path to the core database the updater will upgrade to. This is usually named something like coreX.db where ‘X’ is a version number, sans the decimal (e.g. core31.db). It is generally in the same directory as the updater, so the path is just the file name. This core file will be different for each upgrade.

The oldServer is the path to the "moo" binary that launches the old database, and the newServer is the path to the one that launches the core database. These may or may not be the same file. For instance, if the moo binary were being upgraded along with the database, the old database would require the old binary and new database would require the new version. The moo binary doesn’t change frequently, and there are rarely other reasons the old and new databases would require different binaries, so often the variables point to the same file.

The versionToUpgrade is a string composed of database versions this updater accepts, delimited by the outputDelimiter. For instance, the updater for 3.1 will upgrade a 3.0, 3.0.1, or 3.0.2 database to 3.1. So the versionToUpgrade will be "3.0|3.0.1|3.0.2", assuming the outputDelimiter is "|".

It is important to note that the cvwconnect binary reads the versionToUpgrade string in order to do the version verification. Thus if the outputDelimiter is changed, the cvwconnect source must also be modified to parse the new delimiter. In a similar way, the updater checks for errors from cvwconnect in the file indicated by the errorFile variable, so if cvwconnect is modified to write to a different file, the variable in updater.tcl will need to be modified accordingly.

Last update: 13 October 2000

This work, including the source code, documentation and related data, is placed into the public domain.
The original author is The MITRE Corporation.

THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, AS TO THIS SOFTWARE, OR TO THE ACCURACY, CAPABILITY, EFFICIENCY, OR FUNCTIONING OF THIS SOFTWARE OR ANY DERIVATIVE WORK OR OTHER ITEM MADE, USED, OR SOLD THAT INCORPORATES THIS SOFTWARE. THE AUTHOR OF THIS SOFTWARE ASSUMES NO RESPONSIBILITY FOR ANY CONSEQUENCE RESULTING FROM THE USE, MODIFICATION, OR REDISTRIBUTION OF THIS SOFTWARE.

Information in this document is subject to change without notice.
Other products and companies referred to herein are trademarks or registered trademarks of their respective companies or mark holders.