Does TCP/IP have a future in the data centre?

Posted by Roel Gloudemans on 29 September 2008 | 0 Comments

Tags: network, ethernet, IPv6, tcp, udp

Network Convergence is a hot topic nowadays. This is driven mostly by storage networking. The first products which can deliver Fibre Channel over Ethernet (FCoE) are starting to appear even now. There are a couple of drivers for developing Ethernet for storage networks:

• Ethernet development is ahead of Fibre Channel and Infiniband in terms of speed
  
• Common network layer simplifies infrastructure management
  
• Reduction in the number of network interfaces and thus initial hardware cost
  

Ethernet is a best effort protocol by nature and as such not suitable for storage networking, which requires guaranteed delivery of the data. Furthermore, native Fibre Channel and Infiniband have lower latencies that common Ethernet. To bring all these worlds together, new developments are taking place on the IEEE802.1 standard, which describes Ethernet. These developments fall under the common denominator Converged Enhanced Ethernet (CEE) also known as Lossless Ethernet. Though the first FCoE products are starting to appear, the new standard is not finished yet and the new equipment is pre-standard. CEE is expected to be ready for large scale deployment in 2010.

Average data centre networks use Ethernet. Even for inter-data centre connects Ethernet is used more and more. On top of that the most used protocols are TCP/IP and UDP/IP. If Ethernet is made lossless, one could wonder what the advantage of TCP over UDP really is. A list of functional specifications of the IPv6, UDP and TCP protocols:

IPv6:

1. Can make use of heterogeneous network
   
2. Addressing
   
   1. Address scopes (link local/site local/global)
      
   2. Routing; the ability of packets to traverse from one network to another. The use and usage of routing within the data centre is limited. Routes are static and different sub networks for security, but only in combination with VLANs to provide network separation.
      
   3. Address auto configuration; IPv6 interfaces can configure themselves automatically, starting by querying the nearest router using their link-local address, which is normally derived from the interface's Ethernet MAC address.
      
   4. Multicast; The possibility to address a select group of hosts at once
      
3. IPsec; IP layer encryption
   
4. Prioritization; Used for flow control. Is now well described yet.
   
5. Jumbograms; IPv6 datagrams can be much bigger than IPv4 datagrams. Sizes up to 4Gb are allowable.
   

So how does this compare to Ethernet:

1. We are talking network convergence here. The network is uniform by definition (OK)
   
2. Ethernet only has the MAC address (OK)
   
   1. No scopes exist. Addressing each individual system is a major task. Since the MAC address, for our purpose, is just a random number, some mechanism to manage the MAC addresses is needed. An equivalent to DNS at the very least. (Partially)
      
   2. As mentioned earlier, routing within the data centre is limited. Network bridges can be created. 802.1aq, Shortest Path Bridging, will optimize the network paths (Partially)
      
   3. Each interface comes with is MAC address preconfigured (OK)
      
   4. Ethernet has multi casting capabilities as well (OK)
      
3. 802.1X, Network Access Control and 802.1AR, Secure Device Identifiers can provide part of the functionality. As far as I know no current standard for encrypting the Ethernet payload exists (Partially)
   
4. Prioritization is handled in 801.1p, Class Based Flow Control (CBFC) which uses the 3 bit Priority Code Point from the VLAN header (OK)
   
5. Ethernet can use Jumbo Frames which size up to 9K. Since even the Ipv6 Jumbograms must use Ethernet, the best efficiency from a network point of view will be Jumbo Frames, without any IP at all (OK)
   

Now on to UDP. UDP provides checksumming and application multiplexing. Checksumming is also provided in the Ethernet header. Application multiplexing has no Ethernet equivalent. Application multiplexing (the usage of UDP ports) is a core requirement on networking.

The benefits of using TCP over UDP:

1. Ordered data transfer
   
2. Retransmission and discarding of duplicate packets
   
3. Flow control
   
4. Congestion control
   

These benefits provide nothing new over what CEE already offers:

1. Ethernet relies on in order reception of packets. Each packet will take the same route (OK)
   
2. not needed, since in principle Ethernet should not lose any packets (OK)
   
3. Flow control is handled by 802.1p and az. Per priority pause frames can be send, selectively throttling data flows. This mechanism is probably more efficient than TCP's bandwidth scaling algorithm. (OK)
   
4. is handled by 802.1au, Congestion Notification. Congestion points en route (bridges) notify end points with rate limiting capacity. (OK)
   

So there it is, for local network traffic, using TCP will be overkill and a reduction of network efficiency. Looking at the trend in Ethernet development, more and more functionality from the higher OSI layers is pulled into the Ethernet protocol. The basic reason why we have IP at all is the fact that the Internet is a heterogeneous environment. With Ethernet becoming more and more the by default standard, the need for higher protocols evaporates. Though Ethernet will never cover the full functionality of the IP stack, though I could envision a protocol that is backwards compatible with Ethernet that can, but that will probably not happing within the next 20 years. Network management would be so much simpler, if there were one standard protocol (not a protocol stack) that would give guaranteed data delivery to one or more systems within a given time.