Outage Yesterday

Yesterday afternoon UKFSN suffered a serious outage our main servers. This knocked out all web servers, the main email servers and the RADIUS authentication service.

Broadband, Backup MX service and DNS services were not affected.

The loss of service started at approximated 4pm. Unfortunately I was offline at the time and did not discover the problem until nearly 7pm. Attempts to restore service remotely were not successful and ultimately it was necessary for an engineer at the Hetzner DC to physically power cycle equipment to get the servers back online which was done by about 9pm.

This brought the servers back online however it was still necessary to repair filesystems before services could be fully restored. Between 10:30pm and 11:30pm all services were brought back online.

Please accept my apology for any inconvenience this loss of service may have caused you.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.