Friday, February 09, 2018

NFS downtime on 2018-02-09

NFS (containing home directories, cron jobs, and public_html web directories) was down for a short period of time on 2018-02-09 from 4:41 PM to around 5:06 PM. In this time, files stored in user home directories and all hosted websites (including https://www.ocf.berkeley.edu) were unavailable. Logins were also affected, both in the OCF lab and from outside the lab to our SSH server. Thank you for your patience while we brought everything back online.

Please contact us if you have any questions or concerns!

Wednesday, January 17, 2018

Downtime on 1/17/18

We're currently recovering from a multi-drive RAID failure on one of our servers. Currently logins, MySQL, printing, account creation, and our internal DNS will all not work. We'll post status updates as we bring things back up.

Downtime was on 1/17 from 13:27 to around 18:00 in total.

Note: MySQL and accounts created are being rolled back to a backup as of 2 AM this morning (1/17), so any accounts created since then or database changes will be rolled back.

Update 1/17 15:25: MySQL, logins, and DNS are all in the process of being restored. Database restoration will likely take around another hour, perhaps more, but logins and DNS are both close to being fixed.

Update 1/17 16:22: Logins (LDAP/Kerberos) and DNS have been restored, MySQL is getting closer to being restored, probably will be about half an hour on restoring that. Printing is still a work in progress, but getting closer. Account creation is still down.

Update 1/17 18:09: Everything except account creation should be back up again. We are working on restoring all accounts created today and then we will re-enable account creation. MySQL databases have been restored from our most recent backup (2 AM this morning). We have noticed some lingering issues with DNS that we will keep checking and update here if it stays a problem. Let us know if you encounter any other problems with our services!

Update 1/17 19:58: Account creation has been re-enabled, although any new accounts made today still need to be re-created, but new accounts made from this point on should work fine. We also fixed a DNSSEC issue that was preventing valid DNS responses in some cases. Contact us if you notice any other problems!

Update 1/18 03:21: All the missing accounts have been re-created again, but they will need to reset their passwords, since we purposefully do not log those for security and they were lost in our Kerberos rollback. These accounts can reset their passwords through our website.

Saturday, December 23, 2017

Downtime scheduled for afternoon of 12/24

Tomorrow afternoon between 3-5pm the OCF expects to undergo a short period of downtime to reboot our physical servers in order to apply security updates. We do not expect our services to be unavailable for longer than 30 minutes. Let us know if you have any questions or concerns. Thank you for flying OCF and have a happy holiday season!

Sunday, October 15, 2017

New: bulk-add vhost mail addresses

Starting 10/15, you can bulk-add forwarding addresses for mail virtual hosting by uploading a CSV file.

Wednesday, August 02, 2017

Expected power outage 8/2

Due to a wildfire, the UC Berkeley campus is expecting a power outage around 2:40 PM today. All OCF services have been powered down in preparation.

We apologize for the inconvenience.

Update (9:02 PM): services have been restored.

Saturday, July 08, 2017

Downtime for security updates evening of 7/8

There will be a brief period of downtime tonight (July 8th, 2017), around 9 PM, likely lasting about 15-30 minutes. We plan to apply kernel updates to our servers for recent security vulnerabilities and upgrade one of our servers to the most recent version of Debian stable (stretch). Let us know if you have any questions or concerns and thanks for flying OCF!

Tuesday, May 30, 2017

Announcing HTTPS for web apps

Last summer, we introduced automatic HTTPS for all OCF-hosted websites with virtual hosting, helping keep private data sent to these websites safe from government spies and would-be password thieves. Today, we are announcing the same feature for our web application hosting service.

This enhancement has already been rolled out for existing web apps, and it will come out-of-the-box for new ones. For more info, check out the original announcement as well as the updated FAQ below.

What do I need to do?

Most likely, nothing. Thanks to greatly increased Let's Encrypt request rate limits, we can download certificates and configure your website to use SSL almost immediately.

Will this break existing links to my website? Do I need to update posters with the new link?

No. We will issue 301 redirects to the updated URL.

Are you sending the HSTS header?

Still not yet, but it's certainly on the agenda now that we've been using HTTPS on regular virtual hosts for almost a year. We already send the HSTS header for www.ocf.berkeley.edu, including all websites for individual accounts.

What even is app hosting? How do I get it?

Web application hosting is a relatively new service we offer which allows groups to develop sophisticated websites using modern web technologies not available with regular virtual hosting. The eligibility requirements are the same as for regular virtual hosting, and you can apply be emailing hostmaster@ocf.berkeley.edu. Please read the help page for more information.

If you have any more questions, please email us.

Finally, you can log into your website at ease!