Auntie conquers the internet

The dot com boom left a lingering impression that you need enormous investment and hundreds of staff to develop a successful internet presence.

But guess how many people it took to build a web site that achieves over 400 million hits every month, handles 10,000 concurrent media streams, and was the online news provider the world clicked to in the immediate aftermath of 11 September?

Give up? Until a little more than a year ago, the bbc.co.uk infrastructure with over 400 Sun Microsystems' servers was built, run and maintained by just two people.

'We're on a very tight budget,' says Norman Hitch, head of commercial internet services at BBC Technology.

Hitch was there at the start - he put the first 64 kbit/s internet connection into the BBC in the late 1980s. He saw the operation grow to become the largest content site in Europe, but the focus on expenditure remains. Even now only 23 staff are involved.

The BBC became an ISP in 1997 for access to high bandwidth without renting expensive connections from service providers. The move was vital to keeping costs down - as was the choice of Unix to host the web site.

'When you've got a lot of traffic, you want to manage the hardware remotely. You don't want to sit on it, you want to do it from a mobile phone. The Unix environment allows you to do that,' says Hitch.

Cost-effective support arrangements are another factor.

'If we have a disc failure we take the server out. When we get time, we go and fix it. It's been designed that way because there is a lack of resource to run it. We tend to have spares rather than ongoing maintenance contracts, except for the really critical boxes,' he says.

The servers are managed remotely and run in a 'lights-out' environment at Telehouse facilities in London's Docklands and New York - in the former shadow of the World Trade Centre.

'We wanted to have the disaster standby capability, and 11 September proved to us it was the right way to do things,' says Hitch.

The New York site survived the terrorist attacks for three days, until its generators lost power because fuel supplies couldn't be delivered to downtown Manhattan. Once access was granted, the generator overheated because dust had clogged its filters.

'We were down for about 33 days overall, but we managed to take the content from New York and host it in London because we've got our own ISP connectivity. We managed to serve the whole lot out of London, and the increased demand as well. Traffic was three times the usual level,' says Hitch.

But he admits to an element of luck in keeping the site operational

'We were expecting a huge load for the general election, and had extra boxes in for that. They hadn't been re-deployed at the time. We spent the night of the 11th building these boxes into the servers to serve the news. Slowly but surely we got everything under control. We weren't actually prepared for what happened, but the boxes were available. Now the US has discovered there is a source of news other than CNN,' he says.

Hitch is proud that the principles established in the past proved capable of dealing with such a disaster.

'The number one lesson is no single point of failure. Have your backup live as part of the service infrastructure, and if you can, put it on a different site and load balance between the sites,' he says.

In November last year BBC Online was relaunched as BBCi, to reflect the convergence of the internet and interactive digital TV.

One of its biggest growth areas is streaming media. Radios 1 to 5 are broadcast live over the web, and it has been a new experience for the BBC and its audience relationship.

'If there's a football match on TV and there's a transmission failure you get a few letters complaining. We've been streaming Scottish football league commentary via the internet, which isn't available on any other platform. When we had problems, instantly we had 75 to 80 per cent response from the audience straight back. We have much bigger interaction with the audience,' says Hitch.

BBCi takes that interactivity further. The BBC's recent 'Walking with Beasts' series was accompanied by alternative soundtracks, available to digital TV with supporting material online.

'It's about getting people used to a similar look and feel between the internet and at home. It's a way of getting people more involved in different formats,' says Hitch.

'For Wimbledon tennis we had the most concurrent streams we've ever had, millions of people listening. Afterwards, the audience stayed up there. When new customers come to the BBC they stick, and they come back again.'

The BBC's founders may not have foreseen the internet age, but they would surely approve.

Like this story? Spread the news by clicking below:

 del.icio.us     Digg this     reddit!