Network problems affecting web and streaming services
We are currently seeing issues with out data centre. We are addressing these problems and services will be resumed as soon as possible.
21:20 - UPDATE - All servers appear to be back up and running. We are closely monitoring the servers to ensure continued service. Problems with the SAN caused the problems.
stream1.hippynet.co.uk RAM upgrade
stream1.hippynet.co.uk is currently down for a RAM upgrade. Downtime is expected to be minimal.
UPDATE: RAM upgrade has been successful and the server is back online
Audio server problems
00:06
We are currently experiencing problems with stream1.hippynet.co.uk
Our engineers are looking into this.
00:27
stream1.hippynet.co.uk is now back online. Problems were caused by a connectivity problem which has now resolved. We will continue to monitor the server. If you are experiencing any problems with your stream please contact support.
Network issue affecting web, mail and DNS
On the morning of Friday 14th May 2010 we experienced a network issue that rendered our web, mail and DNS servers unaccessible. Streaming servers were unaffected, but were only accessible by their IP addresses due to the DNS issue.
This problem has now been rectified and systems are working properly. We apologise for any problems.
We will be installing an extra DNS server along side the streaming servers adding redundancy to our network.
Audio Sever issues
We're currently seeing network issues affecting our audio servers. Engineers are currently working on this and normal operations should resume soon. Web and email services are unaffected.
Update
All problems appear to be fixed and everything is running normally again.
SAN Failure affected Web Servers
The SAN supporting the web servers did not recognise a hard disk during replacement, causing problems accessing the SAN. This resulted in a few minutes of downtime. All systems are now working normally. Audio streaming servers were unaffected.
Rack power failure
wizard.hippynet.co.uk suffered from 57 minute of down time during the night due to a power failure in the data center. The backup generator kicked in and picked up 99% of the data center but unfortunately our servers were in one of the racks that were not picked up by the generator. Our engineers resolved the issue and systems are now running normally. We will be moving the web servers to the same data center as the audio streaming servers in the near future.