Author Topic: Outage  (Read 5967 times)

0 Members and 1 Guest are viewing this topic.

Xepher

  • Techsmith
  • Administrator
  • *****
  • Posts: 4,325
  • Illegitimis non carborundum!
    • View Profile
    • Xepher.net
Outage
« on: November 02, 2009, 12:14:55 am »
Update: Second outage (0700 UTC Nov. 3rd) is not my fault. :-P The datacenter lost power and all servers there went offline. It's now back up as of 20 minutes ago.

My apologies for the outage today. The webserver was offline about 12 hours (exactly during the period where I was asleep.) It looks like it tried to restart as per normal due to monthly log rotations, but something prevented it from coming back up correctly. Unfortunately, I don't really know why, as it's an error I've not seen before.

It's back up now, and I'll be keeping a close eye on it. I'll try some troubleshooting later in the night to see if I can narrow down the cause. Thanks to all of you who IMed me letting me know there was a problem... If I'd been anywhere but asleep, I would've gotten to it much quicker.
« Last Edit: November 03, 2009, 09:23:10 am by Xepher »

Xepher

  • Techsmith
  • Administrator
  • *****
  • Posts: 4,325
  • Illegitimis non carborundum!
    • View Profile
    • Xepher.net
Re: Outage
« Reply #1 on: November 02, 2009, 07:55:38 am »
It looks like it didn't handle the "graceful" restart after the monthly log rotation. I've changed that to a hard restart of apache, which should hopefully prevent future problems.

fesworks

  • Administrator
  • *****
  • Posts: 923
    • View Profile
    • http://www.fesworks.com
Re: Outage
« Reply #2 on: November 02, 2009, 02:51:31 pm »
Thanks! :)

Xepher

  • Techsmith
  • Administrator
  • *****
  • Posts: 4,325
  • Illegitimis non carborundum!
    • View Profile
    • Xepher.net
Re: Outage
« Reply #3 on: November 03, 2009, 09:23:20 am »
Updated with news about second outage.

Databits

  • Global Moderator
  • *****
  • Posts: 1,607
  • Programming's not just a science, it's an art.
    • View Profile
Re: Outage
« Reply #4 on: November 05, 2009, 07:49:50 am »
Why would you restart the whole server after a log file rotation anyhow? :P
(\_/)    ~Relakuyae D'Selemae
(o.O)    
(")_(")  [Libre Office] [Chrome]

Xepher

  • Techsmith
  • Administrator
  • *****
  • Posts: 4,325
  • Illegitimis non carborundum!
    • View Profile
    • Xepher.net
Re: Outage
« Reply #5 on: November 05, 2009, 10:39:38 am »
Hmm... I guess I phrased that poorly. It was just the webserver (aka "apache") that was restarted, not the whole machine.