Random Thoughts – Randocity!

Comcast is a home cable provider. They provide internet, video cable and home phone service (over the Internet). Yet, they really don’t know the first thing about how to run a network. Let’s explore.

What is World Class Networking?

To provide high quality, high performance networking, it starts with a datacenter, network gear, network interconnections, 24 x 7 staff and monitoring.  It also requires hiring qualified networking staff that actually know what they are doing and a way for front line support to contact that staff in the event of an suspected outage. It’s quite clear that Comcast doesn’t have this in place. What do they have instead?

They have some sort of networking team, although I’ve yet to determine where they exist. The customer facing piece is the front line (i.e., telephone) support who’s sole job is to roll trucks to the cable user’s home. For customer access, that’s it. It’s always the problem of the home user’s equipment. It’s never ever considered to be a problem with Comcast’s network gear, internal network, DNS servers, interconnections, etc. The front line support’s sole job is to blame the home networking setup of the user for ‘the problem’.

In fact, I have clearly shown them on multiple occasions that it is, indeed, their network gear that’s at fault. Yet, the front line support has no way to contact their networking team to have them check or work on the network.

Monitoring

It’s quite clear that they also have little, if any, network monitoring in place. If they did, they would see these problems and fix them timely. Yet, over a weekend and specifically late into the evening, a problem can persist for literally hours until someone wakes up at 8-9AM, logs in and corrects the issue.  Networking isn’t a 9 to 5 problem. It’s a 24 x 7 problem and staff needs to be ready to act at any time of the day or night.  Treating your network as a 9 to 5 activity clearly says you have no concept of what it takes to run a World Class Network.

Maintenance  and Notifications

As a customer of any other business class network, part of the contract states that they will provide you with network maintenance plans and times and duration of the disruption in addition to a service level agreement.  That is, the minimum amount of service you will get during your billing period. If it goes below that amount, they have to credit your account. Why is it that home users do not get this same level of service? We all know system updates are required. Equipment periodically needs to be rebooted. Configurations need to be changed. Most of these activities require equipment restarts. When this happens, it causes a service disruption. Buying a network connection to XO or Global Crossing, they will notify you regarding their maintenance plans, what their plans entail and how long the duration of the outage.

How does Comcast notify their customers regarding maintenance plans?  They don’t.  There is never ever any notification of planned maintenance. Yet, at least once or twice a week at around 2AM my cable modem gets rebooted or my internet connection just stops working.  It’s never during the day. It’s always around 2-3AM which says ‘system maintenance’. Yet, there has never once been an email sent to my account notifying me of any system maintenance plans.  Again, this does not say World Class Network.

Operating a Network is not a Toy

Comcast treats their customers and their network as if it were some type of playground. As if the customer won’t be bothered by them taking down gear (and the network) randomly without any notice.  Why would anyone in their right mind think that it’s okay to run a network this way?  Yet, here we are. This is the way Comcast has operated its network for years.  In fact, this is exactly how AT&T Comcast ran their network before spinning it back off into Comcast again.  But, that doesn’t make it right. It just makes it amateur.

Worse, they keep their front line support staff completely in the dark regarding these maintenance plans. Literally, you can call up and talk to a rep who will adamantly claim nothing is going on with the network. In fact, they don’t know that the network team is busy reconfiguring and rebooting routers across the entire network taking down neighborhood by neighborhood. It’s only after about 15 or 20 minutes do they finally know something is going on. This is a ridiculous and laughable scenario. Any other network provider would simply laugh at Comcast’s method of operating.

Operating Transparency

It’s definitely time for Comcast to step up into the big leagues and treat both their network and their customers with respect and care. In that goal, this wall between the phone support team and the networking team needs to come down. They need to add transparency between the phone team and the network team so that when there are issues, the phone team knows immediately.  There is absolutely no excuse for the phone team not knowing what’s going on at the moment that it’s happening. Worse, rolling a truck is expensive, especially when the problem has nothing to do with the home user’s gear. Rolling a truck is a whole lot more expensive than picking up the phone and contacting their network team to investigate. Or, better, give the phone team access to the monitoring system so they can actually see the live outages.  No phone call necessary. You would think that escalating to the network team would always be the first step before rolling a truck. You would think that, to save on costs, they would want to ensure that the problem is not internal before suggesting the expense of rolling a truck. I have no idea how Comcast’s management hasn’t been called out on this silly and stupid waste of money.

But no, they don’t care about that. Worse, even if you do schedule a truck, by the time it shows up days later, the problem is already corrected showing that it was, in fact, an internal network problem that someone finally fixed. This is definitely not the way to run a network.

The Time Has Come

It’s finally time for Comcast to offer full operational transparency between all of its teams and the customer. It needs to notify customers of each maintenance plan affecting their network and its expected outage duration. With email the way it is today and the fact that every Comcast customer gets a comcast.net email address, Comcast has no excuse not to slip these notifications into the inbox, just as they do for their monthly invoices.  Comcast literally has no excuse for running their network in this slipshod way for so long. It’s just plain and pure laziness that states unequivocally that Comcast can’t provide and isn’t providing a World Class Network to its subscribers. If you’re a Comcast user and you read this article, you should tweet to any Comcast employee and demand that Comcast provide the same level of service you’d get from any other network provider.

Note: I’m actually having to use my Verizon LTE MiFi card to write this article because Comcast took down the network gear or the DNS servers (can’t tell which) without any notification. I also know that it’s completely fruitless to call into the support team because they’ll simply want to roll a truck, which is absolutely ludicrous considering my cable modem is still fully connected to the network. Trying to explain that to them is useless. They would also run a bunch of useless tests and then only find out later that some random network tech was actually rebooting gear for some unannounced reason.