OrconBlog

One subject which gathers a fair amount of attention for us from some of our more tech-savvy customers is our "L7" box, or the appliance we use to manage and prioritize traffic on our network. We've always been transparent about its use and the fact that we do manage bandwidth in this way - see here for the exact wording on our website - but it's been suggested more than once that there is more to it than we claim. I thought it'd be good to lay out in a bit more detail specifically how broadband traffic is treated on our network; how we actually do it; and what tools we use to do it - in the interests of openness, and hopefully to put to bed some of the suggestions that we're specifically breaking certain applications, in the same way that other carriers worldwide have been accused of doing.

Another important aspect to cover off is the reasons why we do this in the first place - it's a complicated issue, and worth attention. Personally, my instincts are very much again the concept of "Net Neutrality" - the very idea that a large faceless corporate telco would purposefully degrade performance for specific applications is one that has me seeing red. But there are definitely cases for proactive management of traffic across a network to preserve or improve performance for all users equally, as well as opportunity for service providers to add value to content owners that deliver service over their network - see here for one small example.

I'll probably cover some of these topics off in the future in a little more detail, but suffice to say for now, the primary goal of the traffic management techniques that we employ in the Orcon network is to ensure that all users and applications receive a fair level of service, and that a small set of users don't ruin things for everyone else.

So; to technical detail. I'll start off with the equipment.

Our long-standing and extensive use of Juniper network in our network notwithstanding, it may surprise some to learn that the specific box we use to manage traffic is actually made by Cisco, and we've had it in the network for coming up to three years now. Specifically, the Cisco Service Control Engine, or SCE. It's colloquially known as a "DPI", or Deep Packet Inspection, capable device. A normal ("layer 3") router will investigate only the headers of a packet when making a routing decision; typically the destination IP address. And a switch will only look at the destination MAC address of a frame. A DPI-capable device will look far deeper into the packet, inspecting the actual payload, matching it against a database of usually thousands of signatures to classify it as a specific protocol. This means that traffic can be identified far more accurately; while applications can run on any port with a simple client/server reconfiguration, changing the way that the application actually exchanges data over the network, and therefore its layer 7 signature, is a lot trickier.

Of course, this doesn't mean that it's not impossible, and it does in fact happen all the time, particularly with P2P protocols. Which is why devices like this are usually quite easily upgradeable to support additional signatures as applications change over time.

Once traffic has been identified, it can be classified and managed. The SCE is a pretty powerful and capable box; it can prioritize, mark, shape, block, or redirect traffic at very granular level - by individual user, by blocks of users, on a physical link basis, or even enforce policy on a per-transaction basis. We don't get into that level of detail; instead, we aggregate our broadband users into a single policy. The hierarchy of policies is flat, and each has a different priority:

Business bandwidth is generally committed, and has the highest priority. Dialup and broadband come next. It's worth pointing out that this post is specific to broadband traffic only; business bandwidth is NOT managed in the same way. Business bandwidth is simply allocated to a specific customer for them to use as they see fit.

The rules used to classify traffic (remember, broadband only) are pretty simple. We map all traffic into one of three classes:

• "P2P"
• "Default" - includes http, email, ftp, gaming, news, streaming, IM, VoIP, etc.
• "Generic" - any other protocol for which we haven't classified.

The list of protocols that fall into the default class isn't exhaustive; and it's updated all the time. The bulk of traffic by volume is accounted for by those 8 types of applications though. Once traffic has been classified into one of the three categories above, it's then given a priority. It works pretty simply - the higher priority traffic is transmitted first in times of congestion. The SCEs know how large each of our transit links are, and knows when utilization starts to get high. A diagram may help - this is a (stylized) example of a typical 24-hour period for our broadband users:

So the pipe is full at peak, which means that the highest priority traffic ("default") is transmitted first, and the lowest ("P2P"), is slowed down to fit. "Generic" traffic is slightly lower priority to "default", although we operate enough of an overhead that the generic class is virtually never impacted.

It's worth noting another useful side affect - if we ever have a failure on a transit link, the SCE will automagically protect the higher-priority traffic - e.g. business, dialup, and broadband browsing/streaming/gaming/etc.

Some of the statistics are fairly interesting:

• Peer to peer generates approx 25% of broadband traffic by volume.
• Fairly amazingly, streaming video traffic generates another 25% by volume. This marks a big change on a year ago, when nearly 50% of traffic by volume was P2P.
• Most of the rest falls into the "default" class - and most of that is http traffic.
• Approximately 5% of traffic is "generic" - in that there is no specific classification for it. This traffic is made up of thousands of different applications.

Hopefully this has been a worthwhile read for those of you who have wondered what our L7 box really does. There is a copy of this article on my personal blog (here) as well; I'm also planning on following this article up with others, both to offer further insight into the technical architecture of our network, and to explore and explain some of the technical phenomena that make the world of telecommunications what it is today...