Eric Wilson | April 29, 2008
COOKING deep inside the Institute of Electrical and Electronics Engineers are two new standards that will eventually convert many technology systems into pay-as-you-go utilities similar to electricity, phones and the internet.
40Gb ethernet networking protocol will enable information to be flashed twice as fast
In 2010, the IEEE 802.3 Working Group is expected to release an industry-standard 40Gb ethernet networking protocol that will enable information to be flashed between PC servers twice as fast as today's machines can talk to their own processors.
At the same time, 100Gb inter-office and internet backbones will flash data 2.5 times faster than tomorrow's PCI-2e servers will pulse information between their own onboard components.
According to Forrester Research analyst James Staten, such cost-effective bandwidth will encourage many business and consumer applications to move from private computers into cloud computing, made up of widely available online services.
Forrester predicts memory prices, which for the past 20 years have not kept up with advances in CPU, disk capacity or bandwidth, will bring the cloud computing issue to a head in corporate data centres.
"Ninety per cent of the cost of a fully configured server is its memory," Staten says.
"This RAM in the cloud is paid off two to five times faster than in the corporate data centre", he says.
Although virtualisation in corporate data centres makes memory use more efficient, most businesses have "off hours", but a cloud vendor supports worldwide clients so there are no off-hours, since most major providers are already global.
Online service providers also enjoy many other economies of scale, such as in IT management, Staten says.
Steve Garrison, marketing vice-president at networking vendor Force10, says his company has been lobbying heavily for 40Gb and 100Gb ethernet from their inception to help drive these economies.
In his opinion, high-speed connectivity will reinforce the current shift back towards centralised IT services, as bandwidth becomes more cost-effective.
"People aren't going to say they want to do distributed storage, Garrison says.
"It will cause even more centralisation through virtualisation. It's wonderful for IT people to have it all in one room where it's all secured."
The draft ethernet specification foreshadows 100Gb running 1000 times faster than today's typical 100Mb desktop network connections, over distances up to 40km, and that's achievable using only one laser.
Up to 88 different coloured wavelengths could be used, each of which could one day be sent down the same optic fibre simultaneously, theoretically providing 8800Gbps of data throughput.
Nortel Asia metro internet networks lead Ryan Perera says he has seen this all before.
"The move to 100Gb is no different to what happened in the market 10 years ago," Perera says. "Everyone was running STM1 at a capacity of 155Mb.
"After a decade it was becoming unmanageable, so there was a shift to STM16, which was 2.5Gb, providing almost 16 times more capacity.
"Then in the late 1990s it went from 2.5G to 10G, a jump of four times more."
Perera recalls there was a lot of industry talk at that time about the fact that the optic fibres would not be able to carry the extra load.
He says today's links capable of running at 10Gb will be able to run at 100Gb using his company's products.
Gartner principal research analyst Bjarne Munch says, however, that most data centres might never need 100Gb.
"It's more focussed on the carrier space for wide area ethernet," Munch says.
"Last year everything stalled because of the politics between 40Gb and 100Gb. I would not expect to see it deployed in the Telstra cloud for some years."
Munch says it's still early days, since the charter for the standards group was only established in December.
The feature sets in 100Gb systems will be geared more towards telecommunications companies than corporate data centres.
However, Staten says, for some businesses, the affordability of 100Gb ethernet, whether in-house or outsourced, could make all the difference.
"For disaster recovery this is very important," Staten says.
"This will make it much easier over larger distances for more applications."
The key to disaster recovery is data consistency, which in turn depends on network latency (the time it takes a packet of information no matter how small, to physically traverse a network).
For example, the latency on the wire from Sydney to San Francisco is 75 milliseconds, a virtual eternity.
In a poorly written application, this delay could mean a transaction recorded on one system might not have registered with another if a failure occurred at the wrong moment.
Such performance issues require that many disaster recovery sites be located within 80 km of their data centres.
Garrison says this will be achievable relatively cheaply using two 100Gb ethernet links joined by a repeater.
The smarts behind this bandwidth bandwagon have been developing in tech labs for years. The first 120Gb test transmission in 2003 involved firing 12 different coloured lasers down an optic fibre, each flashing at the current 10Gb high-speed standard.
Now, lasers are starting to be flashed at 100Gb of binary code each second, with Lucent-Alcatel claiming line honours in November last year, carrying a commercial payload for US service provider Verizon over a distance of 504km between Tampa and Miami.
However, Garrison insists the real winner of the high-speed race is yet to be determined.
The new 40Gb and 100Gb ethernet standards will be vastly cheaper than the current 40Gb SDH standard used by telecommunications carriers, he says.
These cost about $100,000 a port for 10Gb connections and $250,000 for each 40Gb connection because they use high-powered lasers to go long distances (up to 1800km) without regeneration, Perera boasts of Nortel equipment.
Force10, however, cannot not say exactly how much cheaper the ethernet standard will be, since all the necessary components aren't commercially available yet.
"Let's play a game," Garrison says.
"Let's be aggressive and say in 2010 10G is selling at $1000 per port. The rule of thumb for IT managers is four ports of the old speed should cost about the same as the new port speed.
"So a 1Gb port today costs $500, while a 10Gb port costs $2000. So after a while, in about 2015, it will be $4000 for a 40Gb port."
That is why Force10 expects ethernet to knock the older standard out of the metropolitan arena, including from data centres. Of course, Nortel plays in the high-speed ethernet space too. It recently sold a 10Gb solution to Silk Telecom, which handles work for a number of Australian electricity companies.
Nortel says the Silk system uses "industry-standard ethernet to drive a radical change in the economics of carrier networks", to the detriment of the incumbent standard, which Nortel also sells.
All this sounds good on paper, but Perera warns that not every high-speed system will be suited to the temperature variations or soil movements in the Australian environment outdoors.
This means signal losses are often non-linear in their predictability, and that problem is aggravated when lasers operate at high frequencies.
His company's answer is to shift pulses from a single laser into four phases using coherence detection.
This means four 10Gb signals are sent down the 40Gb link on the same frequency, which makes services more reliable, Perera says. "This is a mature technique borrowed from radio," he says.
"We have designed our technology to work with existing old fibre. Operators won't have to dig the ground to put in new fibre."
For its part, Cisco Systems says its popular 6500 switches will be upgradable to handle 40Gb networks.
The smarts built into many corporate networks for security and end user management are often located in edge switches anyway, closer to end users, Munch says.
This means upgrades to the core should not require upgrades in packet inspection or protocol acceleration elsewhere to match the speedy hub.
However, Munch doubts there is any immediate need for 40Gb ethernet in large organisations, saying most Australian companies haven't even deployed 10Gb networks yet.
However, Dell'oro Group research suggests 10Gb shipments worldwide are increasing and have now reached $2 billion annually.
Cisco of engineering and advanced technology director Kevin Bloch says, however, the increased bandwidth will be necessary to feed hungry data centre processors that might otherwise remain underused.
"The typical PCI bus is 1.2Gb," Bloch says.
"Intel and AMD now have products that are not duel-core but eight-core chips, so you have eight CPUs needing 1Gb each. That's why you need multiple gigabits per second to the outside world."
The bus speed of a PC is the rate at which components plugged into the bus, such as the processor or add-on cards, can share their information with each other.
Garrison says the rationale behind the push for 40Gb instead of jumping directly to 100Gb is to better coordinate light flashes on the network with the electrical pulses in computers, so information flows easily between them at the same rate.
He says it's no coincidence the new PCI-2e computer standard due to ship commercially next year, and the new ethernet standard due for release the year after, are both 40Gb.
"This is the first time the IEEE has tuned the standard to specific applications," Garrison says.
What 100Gb ethernet will do to grab business from the SDH standard, 40Gb ethernet will do to Infiniband, a more expensive system board interconnect used by cloud vendors, he says.
This appears to be what the future is all about.
"Imagine you have a PC and a bus and I unzip that," Bloch says.
"The network is becoming a platform for a bunch of services. Google claims it can do computing for 10 per cent of the cost that you can do it yourself. You don't even need a CPU."
Bloch holds out his own corporation as the poster child, citing salesforce.com, now a Google ally, as Cisco's sales platform.
Perera says, however, Google-style operations have the critical mass to justify some of the hype, but a lot more factors come into play when considering if parts of IT should be moved to the cloud.