> Wittenberg VIII: The most infamous of all Witt incarnations, a perfect storm of viruses and a failed backup caused the Great Wittenberg VIII Crash of 10 November 2001, which wiped out our records of 9-11.
NO, no no.. that's NOT what happened... First of all ZERO Viruses are involved
A quote from my probably never coming book, "The Resident Miracle Worker and this downfall":
The Wittenberg Crash of October 10th 2001
As I mentioned earlier in the chapter “switching to php”, Perl, the language Witt VII was programmed in, really liked flat files.
This means that instead of using a database system for storing its data, it uses plain text files.
For example, here is a message from Ben in one of the old archives (all of it is supposed to be on a single very long line). I removed Ben’s email address:
857|07/18/2001 02:18:52 EDT|R. Ben Madison|[Ben’s redacted email] |RE: a Talossan flag question|> For our flag, what would be the correct vexillogical "blazon"?<p>Per fess vert and gules, I believe.<p>Ben<p><p>|0|||||848||0|wittenberg|129.89.118.61,<b>Ident:</b>[],<b>Host:</b>[]<br><b>Agent:</b>[Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)]<br>|R. Ben Madison|13
If you divide the line with the |, you will find some of the fields:
⦁ 857: that is the post ID
⦁ 07/18/2001 02:18:52 EDT: the time is posted
⦁ R. Ben Madison: the user posting the message
⦁ [Ben’s redacted email]: the user’s email
⦁ RE: a Talossan flag question: the post title
⦁ > For our flag, what would be the correct vexillogical "blazon"?<p>Per fess vert and gules, I believe.<p>Ben<p><p>: the post text
⦁ 0: the number of replies
⦁ ||||: empty fields used for the replies
⦁ 848: the parent post ID
⦁ |0|: another field for internal use
⦁ 129.89.118.61,<b>Ident:</b>[],<b>Host:</b>[]<br><b>Agent:</b>[Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)]<br>: the IP address and info about that IP address
⦁ R. Ben Madison: the actual username (which might have been different from the name)
⦁ 13: the user Id of Ben.
This looks all nice and easy, but it causes multiple problems:
⦁ The separation character needs to be “escaped” because if it is used in the content, it may break the format
⦁ It means that to load a single element, you need to search the whole file, possibly loading the whole file into memory.
In a database however, each field is stored individually, properly protected from each other and you can select which rows you specifically want.
But more importantly, it really rare you have to load the whole database into memory…
Now, when I add a row to a flat file, I open the file in what is called “append mode”. It means that the file is opened, and the pointer is placed at the end of the file.
Then, the new data can be added to the file safely and securely.
But the Wittenberg perl script didn’t do that. Instead, it loaded the full list of messages into memory, and then, when a new message was added, would write the whole file on disk.
Worse, opening a file in write mode erases the content of the file, so even if you don’t write anything, the content is gone.
A few weeks before the crash, users were complaining about the speed of posting message or even of browsing Wittenberg.
That’s because every time a user wanted to load Wittenberg, the full data file had to be read into memory.
But then, on that October 10th 2002, something else occurred… the file became too big for memory just as a user was posting a reply.
That person posted a message, Wittenberg was loaded into memory, failed to load and as such, failed to write.
But the file had been opened and as such, we lost everything. All of the 5556 first messages of Wittenberg VII were gone forever…
My only backups covered messages 122 to 971, leaving 4584 messages lost to us.
I reacted quickly. I create an automatic backup so that the next time Wittenberg would crash, we could have a backup of the state of Wittenberg right before the crash.
There were other crashed in the future, but none of them led to data loss. I would simply archive the backup, trim the older messages and restore the backup so that the show could go on.
I also found a different control panel software for my servers which supported daily, weekly and monthly backups to roll back to a previous day if a crash occurred right before the backups are taken.
I also from time to time proactively archived (removed) from time to time the older messages so that the crash would be prevented.
I had intended to keep full archives from that point on, but to this day, I cannot find my earliest archives.
As result, I have all messages from #7837 (on February 3rd 2002) until Witt VIII was turned in February 18th 2003 (with message #19921), but I cannot find my archive from the period of the crash (message #5556, on Novembr 10th 2001) until the #7837 of February 3rd 2002, other than for a small snapshot of the 15 first messages sent after the crash.
This leaves an additional 2264 messages lost due to careless archiving on my part, though one day, I might find them in an old hard disk somewhere. You would be surprised at the number of hard disk I have stored in my safe (it’s over 40, BTW).
For example, I found the archive from #7837 (01/14/2002) to #10624 (04/02/2002) only recently while writing this book, and another for #8398 (02/03/2002)-19817 (02/18/2003).
For months, I thought that the earliest archive was from #14342 (10/10/2002).
This new find from November 2014 will give us almost 9 months of additional history to study since my original republication of Witt in early 2014!
Show Posts