Mozilla IT

Mozilla Scheduled Downtime - 5/31/2007, 7pm - 11pm PDT (0200 - 0600 6/1/2007 UTC)

We will have a scheduled downtime window Thursday night from 7pm to 11pm PDT. The following changes will take place:

* 7pm PDT (0200 UTC) - CVS upgrade. We will be applying security updates to the libraries contained within the chroot jails used by the cvs servers. No downtime is expected.
* 7pm PDT (0200 UTC) - Planet.mozilla.org upgrade. A new version of Planet (Venus) will be deployed. No downtime is expected.
* 7pm PDT (0200 UTC) - Kernel updates. We will be doing various kernel upgrades which require a reboot. Affected systems/applications are as follows:

- FTP Staging
- IRC
- Mountain View and Toronto phone systems
- All database-backed applications

Please let me know if you have any reason why we should not proceed with the planned maintenance. As always, we aim to keep downtime to as little as possible, but unexpected complications can arise causing longer downtime periods than expected. All systems should be operational by the end of the maintenance window. Feel free to email infra at mozilla.com if you see issues past the planned downtime.

Posted by justin at 7:27 PM | Permalink

Mozilla Scheduled Downtime - 5/24/2007, 7pm - 10pm PDT (0200 - 0500 5/25/2007 UTC)

We will have a scheduled downtime window this Thursday from 7pm to 10pm PDT. The following changes will take place:

* 7pm PDT (0200 UTC) - build-graphs.mozilla.org migration. We'll be migrating build-graphs to better, more supportable hardware. No downtime is expected.
* 7pm PDT (0200 UTC) - download.mozilla.org migration. download.mozilla.org will be migrated to our San Jose colocation. No downtime is expected.
* 7:30pm PDT (0230 UTC) - Kernel updates. We will be doing various kernel upgrades which require a reboot. Affected systems/applications are as follows:

- Mozilla email services (messages should be held until services are back up)
- Anything related to CVS, Hg, SVN
- Talkback
- people.mozilla.com
- quality.mozilla.org
- developer.mozilla.org
- Bonsai
- Bugzilla
- Tinderbox
- LXR
- MXR
- Despot
- Hendrix
- Litmus

Downtime is expected to be 5 minutes or less per machine.

* 9pm PDT (0200 UTC) - Bugzilla update. Minor Bugzilla updates will be put into production. 5 minutes or less of downtime is expected.

Please let me know if you have any reason why we should not proceed with the planned maintenance. As always, we aim to keep downtime to as little as possible, but unexpected complications can arise causing longer downtime periods than expected. All systems should be operational by the end of the maintenance window. Feel free to email infra at mozilla.com if you see issues past the planned downtime.

Posted by justin at 8:56 PM | Permalink

Mozilla Scheduled Downtime - 5/8/2007, 7pm - 9pm PDT (0200 - 0400 5/9/2007 UTC)

We will have a scheduled downtime window tonight from 7pm to 9pm PDT. The following changes will take place:

* 7pm PDT (0200 UTC) - AMO global cache testing/enabling. We'll be testing (and enabling if test go well) our global load balancing and caching with addons.mozilla.org in an effort to improve response time. There is no downtime anticipated, but as we are testing, we could run into unforeseen issues.

Please let me know if you have any reason why we should not proceed with the planned maintenance. As always, we aim to keep downtime to as little as possible, but unexpected complications can arise causing longer downtime periods than expected. All systems should be operational by the end of the maintenance window. Feel free to email infra at mozilla.com if you see issues past the planned downtime.

Thanks,
-Justin

Posted by justin at 11:58 AM | Permalink

We will have a scheduled downtime window tonight from 7pm to 9pm PDT. The following changes will take place:

* 7pm PDT (0200 UTC) - Litmus downtime. We'll be cloning the Litmus VM to bring up a staging box. Expected downtime is 2 hours.

Please let me know if you have any reason why we should not proceed with the planned maintenance. As always, we aim to keep downtime to as little as possible, but unexpected complications can arise causing longer downtime periods than expected. All systems should be operational by the end of the maintenance window. Feel free to email infra at mozilla.com if you see issues past the planned downtime.

Thanks,
-Justin

Posted by justin at 3:23 PM | Permalink