Statistics just finished compiling the web traffic in 2006. This doesn't include any other services such as email or FTP.
Highlights:
324 million "hits" (requests) up from 163 million last year.
54 million actual pages served, up from 31 million.
5.7 million unique IP addresses/hosts, up from 3.5 million.
Over 8 terabytes of data transfered, up from 4.5 TB.
That means every minute of this year resulted in at least:
618 hits
102 pageviews
11 new visitors
16 MB transfered
About 1/4 of data, and around 1/2 the pageviews went to http://www.missmab.com run by Amber. Her visitors also donated enough to run this whole place for half of the year. (And that's only counting the 25% that goes to Xepher.net!) Overall donations were good as well, a few hundred dollars more coming in than needed to pay hosting costs. (Which are going up $10/month starting this month.) Both this year and last year were a slight surplus, and I've been saving the money. I plan to buy/build a new server within the next few months which should result in a LOT more storage (nearly a terabyte) as well as being about 10-15x faster in terms of processing power.
Between a full time email server, spam filtering, PHP scripts on nearly every page, and various automated processes, we're about at the limit of what the current machine can handle without bogging down during peak times. I pay for the hosting out of pocket, and pretty much save everything in the paypal account for "rainy days" as it were. I've had to cover hosting expenses a few times over the past few years with that money, bust most of it has been saved. And since DC Buller made a last minute donation at the end of the year, that actually pushed the savings above the budget for the machine I have specced out. Now I'm mainly just waiting on real life to settle down a bit so I have the time to properly setup a new machine and test it before putting it into operation. As I said, hopefully in the next couple of months here, and then we should have a much better machine.
While I'm setting up a new machine though, I plan to do quite a few things differently in terms of backend. Most of it won't be really visible to you or your visitors, but will hopefully improve security, reliability, and administration ease. To that end, I'd like to ask ya'll: what (if anything) is bugging you currently? Anything that's running slower than you think it should, or otherwise might warrant my attention while I design things for the new system? I personally notice that the forum pages (as well as some other scripts) sometimes take a little longer to load than I'd like, and that the IMAP server is a bit slow to load new messages at times. Anything else I should look at, or maybe specific scripts that seem REALLY slow?
Oh, almost forgot, link to the stats! http://xepher.net/stats/xepher-net-2006/ (Change the year for previous years if you wish.)
Well, that's awesome! I'm glad I can donate a few bucks every once in a while too :)
I owe you a bit now that my first Google payment has come in, by the way.
No worries and no rush. :-)
Hello Xepher,
I'm happy that the previous year was good ... let's hope this will be even better :).
I've asked some time ago to be entered in Phase 2... I don't know what happened...
The reason why I asked was only to be able to add that donate button to my page.
If there will be any donations I don't want to receive anything - I want all to go to you
so other people could enjoy free hosting like I did for more than 3 years now :)
My site (http://www.deltha.ro) only accounted for 1.12 % of the last year traffic
(I think that it has 50-100 unique visitors per day). Maybe some of them will spare
a few bucks... who knows? :)
Thanks again for the free hosting (better than paid ones :)) and keep up the good work!
I do apologize I never got back to you on that, I can be absent minded sometimes. The thing is, Phase 2 was my attempt to narrow the focus of xepher.net to more directly artistic stuff. Your site, while certainly just fine, doesn't fit that end. I've said before, I'm not forcing people out that have been here before, but I'm also not moving them to phase 2. Please understand, it's nothing personal, but it's just not where I'm trying to take things.
As for donations... I appreciate you trying to bring in some via your site. As a phase 1 member, you're under no obligation of course, but if you'd like to just link to the main donation page, that would work just as well. http://xepher.net/index.php?page=donations The newsbox/donation-button are only needed for keeping seperate "accounts" for donations. If you're just planning for it to all go through to xepher.net, the page I just linked does exactly that.
Ok. I've added the link. Let's hope some people will donate! :)
Have you looked into the new processors with the virtual machine capabilities yet? It's kinda hot, as you could also set up users on their own virtual machines with a processor designed for it.
New machine is going to be a core 2 duo, it supports VT. I've got some pretty complicated virtualization stuff planned, but each user is NOT getting a VM. The overhead on that is crazy, and you have to dedicate ram to each VM ahead of time. Besides, it's stupid to have a seperate OS installed for every single user. I am however splitting up some things like the web and mail servers and such.
Okay, real life is good, but busy right now. I've been working 10-12 hour days for the past two weeks, including last saturday. As such, I've had zero time to work on a new server. On the good side though, it looks like the money is about to start coming in, so the work is worth it. It looks like we've already sold the house we started a few weeks ago, and for a fairly hefty profit too. The buyers are putting $10,000 down and signing contracts tomorrow. Once that's locked in, I'll feel a lot better about my finances, and will have no problem spending the paypal account to get a new server. I've been a bit hesitant so far, as I wanted to make sure I'd have funds to pay for hosting, just in case my own finances stayed thin. When this deal closes though, I will make more money in the next 60 days than I did in the past two years combined, and all those worries will cease. I guess what I'm saying is new server is coming, but just give me a little while longer. :-)
How do you see the current statistics?
http://xepher.net/stats/xepher-net/
If you're looking for your personal stats, you go to the account management interface, and there's a "regen stats" button that will provide a link to yours, and generate them from the current logs. Depending on the amount of non-archived log data, it can take several minutes for stats to get compiled.
Oh cool!, Other than "Other" I'm the Top Referring site... whatever that means :P
HAHA! Top queries:
reqs: search term
----: -----------
2861: hentai
just be be known, I think someone else here wrote "Not Hentai" in their Meta tag, and I think I did it too :P
but this search confused me:
reqs: search term
----: -----------
109: å°%8fæ«»
Who the heck searched for that 109 times? :/
Don't forget tentacle hentai and tentacle too. :P
I used to have a "no hentai" thing in my about page, but I think I removed it when I rewrote the section.
I should write "not Miss Mab's Comic. This is NOT DMFA " :P
naaaaaaaaah.
"This has nothing to do with Penny Arcade, MegaTokyo, or Something Possitive"
:D
The referring site thing is because I probably haven't put all your domains into the config as "local" domains. Thus, it sees EVERYTHING from your site as a referral. The hentai thing actually tracked to a "not hentai" page on several sites. The random character searches are virus/botnet things. Many viruses will put some random code like that into pages on machines they've infected, then the hacker giving instructions can easily use google to find how wide his infection as spread, and more easily direct his instructions to the compromised machines.
Always a learning experience at Xepher.net ;)
With the new server, is there a way to view our stats?
I've got all the infrastructure in place to do it, I just never put in a link/button on the control panel to let you generate them yourself. Hmm... lemme rig a temp fix here.
Okay, for now I setup a temporary way to do it. If you log in via SSH, run the command "generate-web-statistics" and it'll run things and put it on your site as /webstats/ That is, http://username.xepher.net/webstats/ It's not a very smart script, so make sure you don't have something else in public_html/webstats/ already, or it may overwrite the index file there.
cool! Thanks :D
Pardon me, but is there any reason why my stats are only showing March the first through March the third?
Oops... default file permissions don't let anyone but the owner read a file. When you run the analysis, it can't read the newer log files that got created under that restriction. I just added a line to my main stat generator that will change the permissions on the log files each night (as new ones get generated) so it should work better now. Keep in mind, each DAY of uncompressed logs takes about 150MB, so I tend to compress them and store them at the end of each month. That means when you generate statistics, it'll usually start at the first day of the current month... sometimes I'm late and you'll see the previous month as well though. Either way though, you should still get results up until at least the day prior to when you run it. Also note that the server runs on the UTC timezone, so times/hours in the stats are shifted about 1/4 day for north american users.
Congrats on the crazy amounts of traffic! A new server sounds exciting. 8D