A massive power failure hit downtown Vancouver this morning, the result of a fire at underground power lines under Richards St between West Pender and Dunsmuir St. Power has been out for buildings and traffic lights around the downtown core. Downtown traffic is hectic at the best of times, and completely gridlocks in the absence of functioning traffic lights. The underground nature of the fire has made repairs difficult and as late as 3pm it was still not safe for BC Hydro crews due to poor air quality underground.
Noticeable across the net was the loss of the Vancouver Peer1 datacenter in Harbour Centre, which hosts popular Vancouver sites such as Plentyoffish, Suite101, Hushmail, and Bravenet. Apparently the building’s backup generators overheated and failed, and there were witness reports of smoke being vented out of the building (which is some distance from where the first). I’ve recieved an update that Peer1 is powering servers back on at this moment, so these sites may (or may not) be back shortly. A thread on the Peer1 forums is being regularly updated. Techvibes Blog is hosted by Peer1 subsidiary Serverbeach in their San Antonio datacenter.
More updates and pictures from NowPublic.
Update: Ian Bell posts an explanatory email from Peer1 in the comments below. The jist of the failure was that the backup generator came online at the moment of failure, and worked for 20 minutes before failing. The generator was water cooled, and as firefighters responded to the nearby fire, water pressure dropped causing the generator to overheat.