Issues with your account? Bug us in the Discord!
Firstones Server issues...
JackN
<font color=#99FF99>Lightwave Alien</font>
in Zocalo v2.0
Getting REALLY slow Server Response, Timeouts and 500 Server Errors in the last few weeks...
Anything we should check into?
Anything we should check into?
Comments
Host info on our server, as of today, 9pm EST (6pm PST):
[quote]
Disk errors surfaced after the last reboot. A filesystem check is being performed at the moment. All services ( http, ftp, etc ) will remain offline until the filesystem check completes. We hope to have the machine back online shortly. We sincerely apologize for any inconvenience this has caused and appreciate your continued patience. Please check back for future updates.
[B]Update Sept 6th 5:07pm PST: [/B] The server has started, and apaches are up and running. If you’re still seeing problems, please let us know ASAP. — Brian H
[/quote]
Got a 500 error on posting that. Looking into it.
Did I say weeks, I meant days... :s
Looks like the culprit might be the wiki, specifically a massive amount of spam on it combined with search engines choking on trying to index that spam.
Therefore, temporarily:
* Disabled new user creation on the wiki
* Disabled editing on the wiki
* Blocked search engine indexing of the wiki
I am going to try to do some mass cleanup of the wiki by editing the database over the next few days to get rid of this spam, then I will re-enable search engine indexing and editing. I will need to investigate if we can do a better job of blocking spam accounts from being created before re-enabling new user creation.
I know there are some defaced pages still floating around the wiki, but it will be a bit more involved to track them down than simple mass editing via the database.
There were around 6000 spam pages, some of them serving images uploaded to our server via MediaWiki, which I still need to locate and cull on disk. 761 images...yuck. All the database entries are gone so no one can find them at least.
Removed several hundred MBs of spam data from the harddrive.
And now it is time for dinner, an hour and a half late thanks to fixing Firstones :p
I thought we just existed in the nebulous of the net...
Jake
There might be something similar for the wiki...
Just thinking out loud.
And do you have to restart the server to clear all that indexed crap in the database out of memory to get the reults?
I've been monitoring the logs for the past several hours. Every time we start getting 500 errors, Microsoft BingBot is busy doing a massive indexing binge. It will query several hundred (or thousand) pages for changes over a couple minute period, during which 500 errors sprout like rabbits for just about anyone else accessing the site at the same time.
I've added a crawl delay configuration directive to robots.txt for Bingbot only which will hopefully slow it down, but it may take a day or so before it actually gets to rechecking robots.txt.
In the interim, not much I can do about speeding it up.
That's exactly what happened in Terminator...