The Race To Zero: RF Considerations in Wireless High Frequency Trading Architectures

We are now within few micro-seconds of the speed of light, we continue to strive to reach the limits of the physics.

In our first two papers discussing the  “Race to Zero” we discussed the advantages of using microwave radio technology in long-haul High Frequency Trading networks and millimeter wave radio technology in Metro High Frequency Trading Networks netwrorks “Race to Zero Metro”.  This paper will discuss the various RF technologies currently in use when architecting a wireless high frequency trading network and their associated advantages and disadvantages.

Introduction:  Ultra Low latency High Frequency Trading networks are achieved by the proper combination of RF technology selection and path planning.  Wireless technologies are emerging as an improved latency solution to fiber optic because of the benefit of the transmission occurring at approximately the speed of light¹ whereas fiber optic or any other wired solution has a media loss penalty of ~33%.  NeXXCom Wireless, as the subject matter expert and thought leader in this application will demonstrate the trade offs and risks with wireless which include availability, reliability, bandwidth, device latency injection, network handoff, and serialization that are necessary to ensure that the benefit of wireless in a low latency network can be optimized.

Wireless is not new, what is new is the development of the digital signal processing necessary to achieve ultra low latency.

With the upswing of financial firms, particularly in the high frequency trading (HFT) application, exploring wireless as a means to enhance their competitiveness, radio manufacturers have entered into the low latency arena; too often their offerings are not clearly represented and this paper will identify hidden latency risks and performance risks. We will attempt to demonstrate how best to architect a low latency wireless network as a turnkey solution.

Wireless 101:  Wireless is not new, what is new is the development of the digital signal processing necessary to achieve ultra low latency suitable for use in high frequency trading.  Microwave communications has been available since before WWII and has been used in all manner of communication systems for over 60 years.  In the market today, most all wireless systems use spectrum from 5GHz up to 90GHz for point to point (PTP) communications. As a rule of thumb, the lower the frequency, the longer PTP distance can be achieved. There are several bands within this spectra that can be licensed so that the PTP connection is protected from interference from other users; there are also many unlicensed bands which are good for low cost, quick deployments but should be avoided for any mission critical communications.

To mitigate costs and time to market, most wireless solutions use commercial off the shelf (COTS) chipsets to emulate the line side of the radio, or the handoff of one radio to another radio or network appliance such as a switch or router.  These chipsets are an effective means of creating a common market radio and deliver a line side interface that operates at OSI Layer 2 (Data Link Layer) and is sensitive to packet size since the entire packet must be processed before routing to it’s destination. Some newer radios add OSI Layer 3 (Network Layer) elements, which adds additional overhead but permits the radio to be a more sophisticated network element.  None of the above are suitable for use in high frequency trading environments.

In order to use wireless effectively in a latency sensitive network, the following considerations are essential:

• The radio must be a true OSI Layer 1 (Physical Layer) device.   To implement this, a non-chipset solution is necessary, which can be done with an FPGA or other custom programmable chip.  This custom implementation will organize modulation, error coding and general signal management².  Non-Layer 1 radios will have more latency per device as an absolute and will have varying latency with packet size (See Table 1).
• The radio must be standards based (essentially a proprietary PHY, but indistinguishable from an 802.3 PHY at the MAC level) to the Internet Protocol (IP) to optimize the network appliance interface and mitigate serialization and/or add data grooming penalties.
• The radio must employ licensed spectrum below 20GHz to allow longer PTP link distances for Long Haul HFT (LH-HFT). This will minimize the number of insertions of radios to the overall network, which will allow the radio network to take full advantage of its speed of light transmission.  For Metro-HFT networks the radio must use 70-90GHz to mitigate serialization/media converter delays.

Table 1 – OSI Layer 2 Latency injection by Frame
Size and Payload through the radio

Network Considerations:  A latency sensitive wireless network will only be successful if the latency of the network carries though to the end user application.  In this paper we also focus on the HFT application in two forms, Long Haul High Frequency Trading Networks (LH-HFT) where datacenters in two remote cities are connected and Metro High Frequency Trading Networks (Metro-HFT) where datacenters within a metropolitan area are connected.   Each of these network models has unique considerations and successful architectures via radio will need to use a separate set of tools and will be discussed later in this paper.

In either case, there are three (3) fundamental areas for latency to be added to the application:

1. PATH Latency – NeXXCom defines the PATH as the sum of the RF trajectory of all of the discrete links between the two end points of the Network
2. Port Latency – This is latency related to port transitions, serialization, encapsulation, media conversion and OSI Layer 2 transitions.
3. Network Latency – This is the total latency of the wireless network, which includes PATH latency plus all the transmission equipment necessary to construct the wireless network.

Figure 1: Basic HFT Network Architecture

To fully understand the essentials of the total latency in the application, you must look at the sum of all three latency components. To reiterate, the overall architecture of LH-HFT and Metro-HFT is similar, however there are some real differentiators that need to be addressed.   Let’s examine each element, the contributing factors to latency and how NeXXCom approaches how we minimize the overall application latency.

PATH Latency should be, in all cases the largest contributor to latency in the overall application (the only way it would not be would be the result of a mismatch in radio technology (i.e. Layer 2 or Layer 3 radio with 100’s of mS level delays).  In LH-HFT networks the PATH factor will be measured in several milliseconds (mS) and in Metro-HFT in microseconds (µS).  The PATH Latency’s most significant contributing factor is design competence.  PATH Latency is most pronounced in LH-HFT where hundreds of PATH miles are in play and because every 10 miles costs approximately 108µS in round trip delay this can make a significant difference.  Port Latency is highly dependent upon the employed radio technology. In order to minimize Port Latency a true OSI Layer 1 radio should be employed to avoid encapsulation and bit stuffing penalties incurred when transitioning from  Layer 2/3 or non-Native Ethernet RF architectures.  All of these latency components sum up to the overall Network latency (PATH, Radio Hardware, Radio Serialization, Encapsulation, etc.).   The Network Latency, which is the only value that matters must take into account all matters of latency contribution as well as the engineering trade-offs, which include latency, bandwidth, availability, reliability and application interface.  Remember if the Network is not working at all, latency by definition, will be infinite.

On the market today are several radio types including NeXXCom’s OSI Layer 1 Fast Ethernet/ GigE systems, OC-3 systems that are Layer 1 “like” and Hybrid OSI Layer 1/2 systems.  As it pertains to the PATH Latency, the central element here is the radio constellation or modulation scheme.  There are some basic facts, again, this is not new science and these facts are well known.  All these radio systems have 28 or 30MHz channels to work with, all have the same power restrictions so constellation and error coding play a big factor into the link margin, which directly correlates to how available the link will be at a given link distance.  The longer the link distance, the fewer links needed to make the Network and therefore the straighter line PATH can follow.  NeXXCom can achieve its OSI Layer1 Fast Ethernet (100Mb/s) at 32QAM, whereas to achieve 135 or 155Mb/s data payloads 64QAM or 128QAM would be required.  With these higher modulation rates approximately 6dB of link margin is compromised, causing a negative distance impact to approximately 90% of the link distance of the 32QAM solution.   Since a typical LH-HFT microwave radio link will have between 13-22µS one way insertion delay, this 10% penalty will be severe and either cause 100’s of microseconds of delay via longer PATH and more radios or induce a system tradeoff to get latency back by sacrificing availability.

We see a 2.7dB disadvantage applied to 128QAM running 135 Mb/s, even with FEC (It would be 3dB worse if no FEC is used).  Assuming antenna and power output are the same, this means that the link distance would be 36% longer for equivalent BER (if no FEC, it would be 5.7 dB worse and the distance would be 92%).  The increased number of hops means increased latency both in PATH and radio equipment to build the Network.

Further to this point is that at the higher data rates the use of IF Repeaters, a device with extremely low hardware latency (typically about 200nS), can not be used as frequently at the higher bandwidth for the same basic reasons.   This creates yet another material impact on latency in tradeoff for more bandwidth at the OC-3 or any non-standard payload above Fast Ethernet.

In Metro-HFT the issue of constellation/modulation is not a much of a factor provided millimeter wave at Gigabit Ethernet is used.  At millimeter wave, the physics of RF propagation/absorption are such that each link will only achieve short distances (2-4miles).   If microwave is used in the Metro-HFT application, the serialization added in the Port Latency (~130µS) is enough to make a wireless solution unattractive compared to fiber optics where the savings is typically measured in 100µS to 300µS latency improvements.  In a NeXXCom Metro-HFT solution a millimeter wave radio solution is used with a blend of modulation and error coding technologies at OSI Layer 1 Gigabit Ethernet.  This unique blend of capabilities allows radios with per device latencies ranging from 10nS to 4.5µS per device; this combination is typically 50% of the latency of the best incumbent fiber solution.  NeXXCom’s Metro-HFT radio suite has a fiber optic handoff so there is no media conversion latency, no serialization to a Gigabit Ethernet switch port and no encapsulation delay since the handoff is Layer 1 raw Ethernet data and not packet over SONET, which any OC-X microwave solution would have to accommodate.

The issue of serialization will apply in any LH-HFT due to the microwave technologies used, which, at best, will be no more that 100-150Mb/s.  With the devices that are off the Fast Ethernet standard, there exist factual issues that create latency penalties.  These issues include:

• Encapsulation of Ethernet over SONET (Synchronous Optical Network).  In this case the OC-X needs to be mapped into an IP environment.  This makes the device non-Layer 1 and this process injects a latency penalty of ~10%
• Layer 2 transitions.  In this case, a radio that is a Layer 2 modified radio (a proprietary framing interface is exposed at the point between Line and baseband framing) is used for the bulk of the transmission network. However, just before the Port Latency point, a Layer 2 device must be used to terminate into the customers Datacenter.  Please refer again to Table #1 and you will be immediately reminded of the variable latency caused at Layer 2.  In this case the Network cannot provide a constant latency to the application and latency will be a function of packet size and include the other penalties addressed above.
• 2+0 Protection schemes to take 135Mb/s and create 270Mb/s.  This will work and will provide more bandwidth however, when two radios are on the same antenna, which is the case in a 2+0 scheme, isolation, noise and other factors that magnify the availability vs. link distance concerns come into consideration.  There will be more bandwidth but either system availability or latency will be the price.

We maintain that the conclusion is evident.  In order to avoid unplanned latency and achieve the lowest latency system it is essential to stay on IP standards to avoid Port Latency and maximize the modulation benefits for link performance in the LH-HFT.  The other way of looking at these tradeoffs is that if all the links in a Network are of equal length and PATH of constant design and NeXXCom’s solution is along side any others, there will be many more occasions where the other system is simply not in operation, or at infinite latency.

Footnotes:

1. The speed of light in vacuum, denoted by the engineering figure of “c”, is 299,792,458 meters per second, or in standard units 186,282 miles per second.  The speed that light propagates through transparent materials, such as glass or air, is less than c. The ratio between c and the speed at which light travels in a material “v” is called the refractive index “n” of the material (n = c / v). The refractive index of glass is typically around 1.5, meaning that light in glass travels at c / 1.5 ≈ 200,000,000 m/s; the refractive index of air for visible light is about 1.0003, so the speed of light in air is only about 90,000 m/s slower than c, or 299,702,458 m/s.

2. NeXXCom Wireless, LLC. has several Patents Pending on the ways and means of implementing ultra low latency radio and radio networks.  The high level concepts are being presented in this paper.

For more information or advice on architecting your Wireless High Frequency Network contact us at:

NeXXCom Wireless  10455 Pacific Center Ct., San Diego, CA 92121  Ph: 619-870-0199

info@nexxcomwireless.com  www.nexxcomwireless.com