Status of OCF Services

We've been working hard to get all our services restored. As usual, we've run into some bumps along the way, but I'm happy to report that we've been able to bring up the following services:

• Webmail: You can read your mail via our webmail interface. However, mail sending and receiving is still in the process of being restored.
• Web hosting: The web server is mostly functional. The OS installation managed to corrupt itself during the move from our old server room in MLK to our new server room in Eshleman. We were able to hobble together a new installation from backup, but we're not sure if all our server components were restored. If something seems out of the ordinary, please comment and let us know.
• MySQL databases: These should be completely functional.
• Shell Access (SSH/SFTP/SCP): We know that many of you need access to your files. Our goal is to bring up our login servers by Thursday night. apocalypse is now online and accepting logins.
• POP3/IMAP: You can now check mail using software on your computer (ex., Thunderbird, Postbox, Outlook, etc.)
• Outgoing Mail: You can now send mail from the OCF.
• Incoming Mail: Our mail infrastructure is split into many pieces that are hosted on multiple servers. Unfortunately, all the pieces need to be brought up before we can re-enable mail (otherwise, there would be a flood of spam, bounces, and incorrectly delivered mail). We hope to have incoming mail restored by the end of the week. Incoming mail is up as of Tuesday night.

Once again, we apologize for the extended downtime. We're exploring better ways of handling extended outages in the future, but we want to emphasize that this downtime was really an anomaly. A perfect storm of factors came together: a long power outage, a server room migration, a network disconnection, and an IP space change.

We'd also like to thank the wonderful people at Physical Plant, IST, Residential Computing, and other campus organizations for coming to our aid. Also, thanks to our users for their supportive and critical comments (hey, we deserve it!).

UPDATE (28-Aug-09, 9:30 AM): We had a bit of a hiccup in our services yesterday night. All services that were working yesterday morning should also be working now.

UPDATE (29-Aug-09, 7:00 PM): We're still restoring our mail services. We haven't forgotten about our other services (printing, web file access, etc.) We're just prioritizing our most important ones.

UPDATE (1-Sep-09, 9:00 AM): We're starting to ramp up incoming mail. Once we're sure that it's working perfectly, we'll begin restoring the rest of our services.

UPDATE (2-Sep-09, 8:00 AM): Incoming mail is completely operational. We're starting work on restoring the rest of our services.