Since about 10:30 this morning, we've been plagued by site slowness (and a server crash).
We've fine tuned our current Web server as much as we can.
The next step -- which we're taking -- is to spend more money and upgrade our server.
The frustrating part is, The Batavian, while popular locally, doesn't require many resources. Our current server should be able to handle the load just fine. The apparent and probable problem is there are a number of black hat Web crawlers (software programs that visit various Web sites sucking in the content). Black hat crawlers don't obey the ethics of how to crawl a site, they won't let a server administrator block them, and they consume a tremendous amount of resources.
Based on the usage pattern we're seeing, we believe that doubling our server resources will give us a better ability to handle the load and greatly reduce the number of these periodic site slow downs.
The upgrade should be in place in about 20 minutes from now.
I was going to sent a tweet
I was going to sent a tweet in your direction asking if there was something wrong with the server, or if it was just me.
I guess it's kind of a "good
I guess it's kind of a "good news, bad news" thing, it's good you have a lot of ads on your site, but there's a lot of indirect contact they bring that is probably inviting these web crawlers to slip their tentacles into your site. Oh well, cost of business. I'm hoping it'll get you more exposure, which brings more hits and brings more ads for you.
I don't think the ads have
I don't think the ads have much to do with it. They're served mostly through javascript, so the crawlers, effectively, wouldn't see them. It's more a matter of we produce a lot content (between stories and comments). Some of these spiders are looking for unprotected e-mail addresses for spammers (never put your e-mail address in plain text on a Web page); others are looking for content that is going to have good search rank in Google so they can steal a portion of it, and who knows what else.
must be the day, my work
must be the day, my work server wnt down too
I was wondering-most times
I was wondering-most times when I try to navigate The Batavian on my laptop w/ Windows 7,sometimes the site freezes,or else the explorer toobar constantly flickers as it's looking for sites......when I use the other laptop w/ Vista,I never have a problem. This has been happening frequently for about last 3 weeks.... right now it's having no trouble at all....
We've had off and on problems
We've had off and on problems with maxing out our server resources. When this happens, the effect is when you first hit the site, it takes a long time to make the connection. Once the download starts -- at least for me -- it goes very fast. I'm hoping this is the end of that.
I don't know if it's related
I don't know if it's related or not but there was an issue with the router and bandwidth at Millenium that started last evening and lasted well into this afternoon. We were without our website until a couple of hours ago.
That would only affect sites
That would only affect sites hosted by Millennium.
Thus why I posted; "I don't
Thus why I posted; "I don't know if it's related or not...."
I havent noticed an
I havent noticed an improvement at all so far. I thought it was all the ads too. There have been a few times I couldnt load the site after several minutes and gave up.
Going into this, the business
Going into this, the business plan was always to have lots of ads on both sides of the page (and I want more!!! <g>), but I've watched for load issues related to the ads.
Once the server delivers the basic structure of the page, it loads pretty quickly. Certainly, the content of the page is there before all the ads finish loading, so you can start reading.
What you see when we're having server trouble is that initial page load stalls. The server just doesn't send even that first bit of data. That's the server having trouble, not the page itself.
Since yesterday, I've noticed an improvement, though it's still not as instant as I would like. It will be interesting to see how things go when traffic cranks up (as it usually does) in the mid-afternoon time frame.
And as soon as I write my
And as soon as I write my last comment, my last few actions on the site have been lightening fast. I like that.
I had something called Pando
I had something called Pando media player installed late yesterday. Since I got rid of it I do notice an improvement ! I didnt realize this till I went to a few other sites.
Howard, for me, logging on
Howard, for me, logging on and accessing the comments section is still taking an extremely long time to load - both at home and here at school. And I am seeing a lot of "waiting for pubads.g.doubleclick.net. at the bottom of the page. It goes on for about 30 seconds. When it finally stops, the page loads instantaneously.
That's a little strange, Bob.
That's a little strange, Bob. What are you using for a browser and on OS?
Because of the ads are being loaded via JavaScript from a third-party server (Google, so always very fast), the rest of the page should be completely loaded, even though they still might be loaded. That's my experience with FireFox, Chrome and Safari.
XP at school, windows 7 at
XP at school, windows 7 at home. I have tried with both explorer and firefox on both platforms. No difference. I don't have a clue what's causing it if other users are not experiencing the same.....