New Server Coming Soon

[Xepher]

On the way to the new server, I've created new SSL/TLS certificates for the mail server(s) and web server. They are, as before, self-signed certificates, so your browser and email client are likely to warn you that there's no authority for the certs. Since they are merely verifying that xepher.net is, in fact, xepher.net, and don't apply elsewhere, it's safe to accept them permanently, and you won't be bothered by it again.

On that note, all mail services now support TLS (it's like SSL, but inside regular channels) so you can set your mail client to use TLS for enhanced security on imap, pop3, AND smtp (outgoing) connections. That way, all mail transactions (including passwords/logins) will be fully encrypted. All this is in place already, since the old certificates expired last week, I had to update the current server anyway.

[Xepher]

So, a question for you all. Would anyone be upset if pop3 email access went away? All the "brilliant" spam filtering stuff I'm doing relies on being able to sort and scan mail in various folders (aka "Junk" and "Not Junk" like things) but pop3 provides no way to download/check/teach anything in those alternate folders. I just grepped though the last week of mail log files, and only three users have logged into pop3 in that time. My guess is it wouldn't be a big deal, but I figured I'd bring it up here for discussion before I commit to anything.

As for a status update... spent most of the day tinkering with the mail system again. I've pretty much got it finished, with all sorts of nifty config options able to be implemented, and that server-side spam "learning" thing I mentioned before. A lot more features I'm glad to be getting into place as well, but I won't nerd-out on ya'll again. Suffice to say, I've just about got the framework in place, so I know what user settings I'll need to collect via the account confirmation process. I hope to have that up fairly soon, maybe the monday, but at least next week. I'll be moving the week after that, so it'll be time where I can't be online much, but people can have the time they need to get their notices and confirm their accounts.

[griever]

I wouldn't be upset...if it cuts down spam, it's worth it, IMHO.

[dragyn]

Wouldn't bother me a bit.  Go for it.

[otrstf]

I'm only using the web client anymore, so POP3 could go. (even though I have much better luck configuring POP3 vs IMAP, I gave up trying to make my Evolution client at home handle both.)  

PS: whatever you did to my spam filters worked!! Many thanks ^.^

[Xepher]

Glad to here it. Even just a few years ago when I put this server in place, spam was more of an annoyance than the plague it is today. As such, I took the design idea of spam filtering being an optional luxury, rather than a necessity. Now half the email accounts here are unusable because the "luxury" wasn't turned on or enabled by a lot of people. Now that I've had plenty of time to use and evaluate the filting programs, I know how reliable they are (and aren't) so I don't feel bad making spam filtering on the new server non-optional, just adjustable and trainable. Hopefully everyone can actually use their email when they need it after the move.

[Databits]

I can write a really easy tutorial on how to use Mozilla Thunderbird for IMAP connections. It's actually pretty easy to do.

[Xepher]

Data: If you get a mind to do that, it could be helpful, but hold off for now, as the new server's gonna function a bit differently.

[Databits]

Where I don't really see how imap would function any differently on the client side of things despite how the server is set up, no problem.

[Xepher]

Pop3 merely allows users to download messages from their inbox to their local machine. All further processing/sorting/folders/etc are handled on the client machine. IMAP keeps most things server side, especially folders. What that means, is that I can create spam filters that are safer, because they can put "iffy" messages in a folder, rather than just deleting them. On top of that, the IMAP server has a nifty hack in it that allows it to run a program when messages are moved/copied into certain folders. What that allows, is for spamassassin's learning programs to be run on messages users decided are (or are not) spam. That is, if you drag/drop a bunch of spam from the Inbox to the "Junk" box, it will learn from those messages and be able to catch more spam in the future. If the same thing was done with pop3, there server wouldn't be involved at all, since the user would just be moving messages around between folders on their own computer....

And wow, I'm an idiot. I just realized that's not at ALL what data meant. I'm not deleting it though, as it's a good explanation for everyone anyway. To answer what I believe DATA's question was (related to doing a tutorial): The new server is using maildirs instead of mbox files. This particular server (courier) handles folders differently than the current server. The current setup requires folders that are totally independant, and you can't make subfolders (not if they have mail in them.) The new server allows for mail and folders both inside other folders, but also has a different namespace layout, using INBOX as the default namespace root... meaning if you don't setup your client right, all folders LOOK like they're subfolders of the Inbox. Under advanced server settings in the thunderbird config, there's an option for "IMAP Server Directory" which, if set properly to "INBOX" makes folders behave as before (and how I think most people would expect.) Since I'm going to be trying really hard to make things understandable to everyone, I don't want to have some people setup with their folders different, and later getting confused when I refer to a certain folder in the spam filter tutorials. (Yes, I know, I'm probably being pedantic.) Also, courier allows a default set of folders already subscribed to (on the server end) and I need to do a few tests to see how different clients handle that. Squirrelmail in particular borks on it, because it doesn't take the list from the server, but from it's own config... it doesn't see a trash folder in it's config, tries to make one, the server says "no, it's already there" and then squirrelmail just gives up.

Anyway, I just think there's enough potential for quirks to pop up, that if you're going to put the effort in, it might be less wasteful if you wait for the actual setup. Also, it gives you a chance to let me know of any bugs you come across in the new system. :-)

[Xepher]

Just a quick update. I'm still moving along with things, but real life is hectic at the moment. I was supposed to be moving into a new apartment yesterday, but hadn't been able to get in touch with the new landlord for more than a week. Finally she called me today, and I'm getting a different apartment, and I'm not getting it until next friday. As I was supposed to be out of this apartment next thursday, it makes for a complicated situation. Once all this is settled though, I should be able to focus on the server again. Sorry for the (seemingly eternal) delays. :-)

[griever]

Uh oh...that sounds like a tricky negotiation between the two...I hope you find something that works for you.  Maybe pre-loading a moving truck/trailer and then just zipping over Friday morning?

[Xepher]

Just a quick status update for everyone. I'm in my new apartment and I love having space to myself. It's great! I've been running a live test of the new anti-spam system, using dspam. It's about 93% accurate, with zero false-positives so far. It's a 100% learning based system, meaning it's effectiveness should get a lot better as you train it. Spamassassin (the current/old system) catches a few percent more spam... but I had numerous false positives though my time using it, and it still let some spam though. It's also about 10x as resource intensive as dspam. Anyway, point is I think I'm going to switch to dspam for the new server, so that phase of testing is pretty much done. People are also going to have the option of using their email here or not. This time, it'll be totally disabled if not in use, before, there was only the option to forward it, which was fine until the advent of spam. You'd think I was born yesterday, the way I set up some of that stuff before. :-)

Most everything else on the new server is running as well as I would expect. All I've got left is to rewrite some of the account management scripts to modernize things. Right now, I'm working on the script that'll let everyone verify their accounts for transfer to the new server. It's times like this that remind me just how poor of a programmer I really am. I'm having to read the manual for every function it feels like. :-) Anyway, I hope to have this working in the next couple of days, at which point I'll email everyone and ask them to verify their accounts. Basically, you'll be giving a valid (offsite) email, and checking/setting all your account/site information. Also, you'll be required to (re)set your password, so it can be re-encrypted for the new system. Passwords are now going to be checked for cryptographic strength too. I know it's kind of a pain, but dictionary attacks comprise about 2/3rds of the hacking attempts on the server, and all it takes is one person with an easy to guess password, then I'm stuck cleaning up the mess.

Also, I have the intention of doing away with the phase 1 and phase 2 account distinctions. I believe there will be VERY few phase 1 members left after the move, and most of those will hopefully have no problem moving to phase 2. If there's a couple that have issue with the newsbox requirement... well, I'll deal with it on a case-by-case basis. Point is though, everyone will have full access to all features.

Lastly, the newsbox. As I've mentioned, I want to revamp/redo it. It needs to accomplish four functions. 1) Get people to notice that a given site is part of xepher.net 2) Promote other sites here 3) Attract donations 4) Allow emergency "broadcast" announcements. I'm going to do away with the regular/boring "news" items in the newsbox. It'll only be used if there's some critical, site-wide message that needs to go out to most everyone. As such, that last requirement doesn't really need to be fit into the newsbox design, as I can have it add a big red box or some such when needed.

Now, on that though, I'm willing to do just about anything that accomplishes those three goals. I want ideas from ya'll, about what you think would be good. Featured sites need images, not just text, I think that's a given. Beyond that (and what I mentioned already) I don't have any ideas yet.

[griever]

Passwords are now going to be checked for cryptographic strength too.

Will you also be requiring people to change their passwords every x-many months?  I know you didn't mention this before, but I was just wondering if this was part of the cryptographic strength.

Also, are you doing away with the counter?  I notice you didn't mention it in the newsbox portion.

I think that the donation button should be on the same line as the title and possibly larger.  I think it might draw more attention at the top, rather than the bottom of the box.  Or, alternatively, have the button be separate (and graphical) and let users position it.  With all the content in the newbox as it is, the donate button is overshadowed.

A few thoughts on the network of sites:
-A series of links to a category page that would then list the site with the description
-A gallery-type list of sites
-What DeviantArt does - a category dropdown and then a list of all the stuff in that category.

[Xepher]

Passwords: No, I won't be requiring them to be changed regularly. That's not much use in a geographically diverse population. It makes a lot more sense in local/corporate settings, where people are likely to "loan" their password to someone they shouldn't. I'm not worried about that, just totally unrelated/unknown hackers being able to brute force a password.

Counter: Probably get rid of it, though I'll still keep statistics with it behind the scenes.

I actually thought about this a lot last night. I'm thinking of doing a very javascripted box. A set size, with an obvious xepher.net logo or something across the top, and the rest of the box will be a picture/ad for the featured site. That picture will fade (javascript, not flash) to a new/random site/feature every 20 seconds or so. When people mouse over it, it'll give a text description/plug for the site, and forward/back buttons to flip through other ads. Mousing over the XEPHER.NET logo at the top will give the "free hosting for free thinkers" slogan, and big options for "browse sites" "hosting info" and "donate." I'll also be mixing in "please help support us..." type ads in with the random rotation of featured sites.

Doing this with javascript and interactivity let's me keep the newsbox a reasonable size AND keep all the aspects it needs, yet not crowd it. The downside (to a few people) will be that it won't really be configurable anymore. It's going to be a set size/color/shape/etc. so people will have to redesign their pages a bit. I'm gonna look into the options, but I think I can make it run in pure JS, and not require pages to be PHP. That should make things easier for people that don't otherwise use PHP.

Another thing I thought about was the donation system. Honestly, we've gotten jack-all from the newsbox, and having this complicated system in place to share donation income is rather cumbersome. I'm thinking it may be better to not accept donations on behalf of users anymore. The thing is, the few donations that do come in that way are usually quite small. Small enough that it seems downright stupid to be splitting 5 dollars, and letting paypal take a cut twice. What I think I may do (and I really want some opinions here) is just ask users to support xepher.net as they feel is worthwhile, rather than a mandated 25%. Users that want to take donations can take them directly, and I'll just rely on good will that they might pass on a fair share. Conversely, I'll still be asking for donations though the newsbox and the main site pages, so some people will hopefully support the network as a whole too. How does that sound?