New Server Coming Soon

[Munerift]

...wow... I thought I had been having bad luck..!!! (at least I've got my laptop and my home PC! )

[Xepher]

I still don't have a date yet for the transition, since I'm still working on the new setup (software wise... the hardware's working great now) and don't know how long that'll take. As I said before though, I plan to give everyone lots of notice before the move, so they won't be surprised by it, and can prepare as needed. My plan for giving notice currently involves this:

1. I plan to email everyone at their xepher.net email address. If you're in Phase 2, there's no excuse, as you should either be checking that, or having it forwarded to a valid account. The email is likely going to include a link for you to click on that will verify your account as still active/in-use.

2. Accounts I don't get a response from... I'll look over the list, and ones I recognize or know something about, I will try to contact them myself or for people that are pretty active here, I may just whitelist directly. I'll probably look at web traffic and see if they're even being visited or not. Ones I don't whitelist will have their accounts locked with a notice page explaining the need to contact me.

3. Any accounts still not responding will not get fully migrated to the new server. I will (and already have) put a copy of all user data on the new server, but I won't actually add them into all the system files until they contact me. I'll probably keep that data around for at least six months, maybe more, just in case, but I'm probably gonna want a good excuse if you takes you a month or more to notice your site is missing.


So, to make this go smoother when the time comes, I ask all of you in Phase 2 to make sure your email is properly set up. Either forwarded to a valid address, or that you check it here at least weekly. Wouldn't hurt to send a test message to yourself @xepher.net to make sure it gets to you. I'll be sending an automated email from adminATxepherDOTnet for any of you who want to setup email rules/filters or such. That address is already whitelisted for the xepher.net spam filters. You may want to do the same if, like me, you use jihad-like levels of additional spam filtering.

[Xepher]

I've got apache (the web server software) running on the new server. I've been doing a lot of testing and tweaking to see what's the best way to set things up. I tested php with the "regular" method we've been using here on the old server for a while now. The new server could run about 70 requests/second and that would full peg both cpu cores. I then tried fastcgi, and that was a huge improvement. 250/second, with only about 25% cpu usage. Finally I tried a new method/module where apache actually spawns a new process for each user. This also hit about 250/second, but only about 20% cpu. I tried a simpler test script, one that just echoed a single line. It still maxes at around 250/second, but cpu use was only 4% or so. The bottleneck isn't the CPU at this point, it's the network and the script itself. Still, the method change alone is about 4-5x faster, and uses about 90% less cpu time. On top of that, the new server (compared to the old one) is about 15 times faster. When I get this thing in place, we're going to have plenty of room to grow. Only limit is going to be bandwidth.

You may be wondering why the new method if it's not that much faster than fastcgi... Well, quite simply because EVERYTHING served from a user site can be served by a process running as that user. No need for fastcgi wrapper scripts, suexec and weird permission settings on php file. I can actually setup php as an apache module, pretty much building it into the server itself, yet it's still just as secure as what we've been doing. No hoops to jump through to make php files work. On top of that, you can secure regular files even more. Since the server itself runs as the user, you can set all your web files to be readable only by you, whereas before, they had to be readable by everyone in order for the server to get to them. It's simpler than before, faster than before, and more secure than before. That sort of thing comes along so rarely in this stuff that I just have to say it once again...



... and no, you don't have to have understood any of what I just said. I've just been nerding it up for the better part of the day, and I needed to rant my success. :-)

[fesworks]

Ok, you say the only limit would be bandwidth? But does that mean its about the same as before?

[Xepher]

Well, for the past week, bandwidth has been horribly lacking, as the datacenter's been having problems with their equipment. Normally speaking though, bandwidth is not the bottle neck. The server is routinely pegging out the cpu during peak hours, and that causes delays when loading pages... especially dynamic pages like forums and such. I believe the new server, combined with this new way of handling such dynamic pages, will be fast enough that the next bottleneck we run into will be bandwidth. That's a ways off though, and things would have to grow quite a lot for that to happen.

To give you an idea of what sort of bandwidth we have, I've clocked downloads to the server at over 7 megaBYTES per second. That's over 50mbps, compared to most home dsl/cable at around 1.5mbps, and dial-up at 0.056mbps. Of course, with this datacenter we don't have guaranteed bandwidth, but that's one reason it's so cheap. Still, our average user right now is about 3mbps over the course of a day, with peaks around 5mbps. That's at least 10x room for growth. I think the new server can comfortably handle us up until that point. If we get larger than that... well, frack, we're gonna need some serious income, and find a datacenter with gigabit hookups. :-)

[Munerift]

...all this geek-speak is getting me all warm and fuzzy inside.

[fesworks]

Touch Fuzzy, Get Dizzy

[dragyn]

Hey, a reference I actually get!

(The image won't load for me, for some reason)

[Munerift]

LOL!

[dragyn]

...and now the image will load, too!  Yay!

And I was right!  I did get the reference!

[fesworks]

hahaha

[psychobob]

ooo

I'm a little techno illiterate, but yah...

One thing though...My Xepher e-mail is CLOGGED with spam, so I don't have any redirecting to my personal e-mail Jenscripter(@)yahoo.com . Will all the information you'll be e-mailing will also be on this thread?

what happens to the old server btw O.o?

[KuroWorks]

Hey psychobob you got coco on you frace

[Xepher]

I will be posting MOST of the information here, as far as news and such goes. It's still going to be important to get the actual email though, as each one is going to contain a verification link unique to your account. It'll be a lot like the confirmation emails you get when you register on a new forum, and will help me automate the task of migrating all the account data to the new server.

As for spam, it looks like you've deleted most of it recently, since your inbox is really small. To keep it from piling up again though, you should turn on spam filtering.  You need to set "Spam Control" to "On" for the filter level to have any effect. I notice you've got the level set to strict (1) but it's not actually turned on... that would definitely explain the spam you're getting. :-)

[Xepher]

Well, I woke up with a not-so-happy email this morning. The SMART (hard disk monitoring system) daemon emailed me saying that one of the drives in the current server has failed a self test with a read error. It's the drive currently housing the root filesystem, not user data, but if it dies, the server is going down. Now, before anyone panics, these things happen with older drives, and things will keep getting worse, but it doesn't necessarily mean it's going to die tomorrow or anything. I feel pretty safe in assuming it will hold out until the new server is in place, but I wanted to give you all warning just in case I'm wrong. I'd suggest (as always) that you make sure you have current backups of your site/email/etc. just to be on the safe side. I've got a backup of everything I pulled to the new server about two weeks ago. I'll be updating that again in a few days again, as I continue getting the new server ready.

In the event things really go bad though, I'll obviously be unable to contact anyone though these forums or my normal email at xepher.net. Anyone that needs/wants to can contact me on AIM via "Xepher42" or at email me at gmail, where I'm also Xepher42. Here's hoping that's not needed though.


EDIT: I'm syncing all user data as we speak. I'll try and do this every couple of days or so until the new server is ready to go... just to be safe.