At about 3 p.m., we noticed a problem with the Web site. The wrong headlines were appearing on stories. The wrong comments were appearing on stories. If we tried to edit a story, the wrong edit screen would open.
All of the content of The Batavian is served out of a database. Databases have segments known as tables. Different tables perform different functions and are related to each through numeric sequences. One of the tables of our database stores revision history. This table had become corrupted, which caused the content to get scrambled.
We don't believe any malicious act caused this, or even human error. It just happened.
The most recent available back-up of the site was from 10 p.m., Thursday.
It took a great deal of time to first figure out the problem and then look over our options for recovery. We decided to back up some files on the server, back up the back-up database and then install everything on a secondary server and make sure that the restored database wasn't also corrupted.
We then had to restore the primary server.
All-in-all, the site was offline for about seven hours. It came back online at 10 p.m. or so while I was at the Cal-Mum vs. Le Roy football game. Then we had a house fire on River Street, so this is my first opportunity to post something and fill you all in on what happened.
The worst of the news is -- besides being down for seven hours -- is that everything posted on Friday was lost. The good news is, I was able to manually save copies of all the home page posts from Friday, so my next task is to repost those. From the best of my memory I'll time stamp the stories to the approximate times originally posted.
Unfortunately, all of the comments left on those posts are gone. Any obituaries, events, announcements, etc., posted on Friday are gone, too.
We apologize for the inconvenience of all of this. It wasn't any fun for us, either. It reminds me of the time I was a reporter at a daily paper in California and the press broke down just as it started the afternoon paper's run. The sense of helplessness feels the same. It's an unhappy situation, but tomorrow's a new day.
I'm sure it was frustrating
I'm sure it was frustrating to figure out! I did see the site when the headlines didn't match the photos, and one sort of hit my funnybone. I did a screen print because I knew it was just a fluke that things weren't matching up.
The headline was "Todays Poll: Which of the top two VP candidates would make a better president"
The photo was TF Brown's Fantasy Football challenge, and this week's challenge was, "Which team defense will surrender the most yards?"
Seemed sort of funny given the hot mess of a VP debate Thursday night!
Thanks for fixing the site, I missed it!!
Dang Technology. Welcome
Dang Technology. Welcome back! I'm glad you got it worked out.