News:

The anti-spam plugins have stopped being effective. Registration is back to requiring approval. After registering, you must ALSO email me with your username, so that I can manually approve your account.

Main Menu

Internal server errors

Started by reinder, July 16, 2007, 08:34:08 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Mithandir

well I've commented the line out again, I'll do some performance tests on my own server using the integer field instead. The goal of that query is to tag the hit logs from spammers so they're not mistaken for real people in the logs.
I'm currently archiving the logs for january->may, but it seems reinder got a lot of traffic these days so it's rather slow (it doesn't seem to slow down the rest of the site though).
Once the archiving is done, I'll kick out the detailed data for jan->may which should free up some 150 Mb worth of data or something. I'll need to rebuild the table at that point, though, so I'll do it in small bursts so as not to inconvenience anybody.

Mithandir

Alrighty....
I've cut the logs table by half for now, deleting the data for december last year and january this year (reinder's two busiest months). It's still big (420 MB + 100 Mb Index), but a lot better already. I'll clean out some more months at a later time but want to monitor how things go first. The table was optimized again and everything seems to be running smoothly, as near as I can tell.

The spammers are still at it, with 155 spams from 115 ip's blocked so far today, but these spammers are running into the first defense (which is the least processor hungry) instead of the content-based filters (like the earlier spam attack did).

Xepher

Yeah, like I said, I think if you optimize that one query somehow, either with indexes, or something like keeping a separate table for known spammers so it's quicker lookup/update than using the main log table then it won't be a problem. I don't mind the storage space for the large logs. Soon as we jump to the new server, there should be disk space to spare. (It's a raid5 setup with 960GB of usable space.) Lemme know if there anything I can do to help you optimize or figure things out. That slow-query log function that mysql has is quite handy, I'd never used it before.