Don’t settle for poor quality of service for IP voice: demand better from your carrier

After 100 years or so, telecom network engineers figured out how to make business phone systems run flawlessly over the public switched telephone network (PSTN). We took a high “quality of experience” for granted. And if it wasn’t there, businesses invoked service level agreements (SLAs) and demanded, and got, compensation from carriers.Audio quality was consistently good. Calls rarely dropped. Latency and jitter, the twin bugaboos of IP telephony were, respectively, a thing of the past or nothing anyone had thought about yet. Now a new era of unified communications has begun. All modes of connecting people – voice, video conferencing, Web conferencing, e-mail, instant messaging, social networking, presence – are moving to the same IP network. They’re integrated and frequently all managed by the same software, by products such as Microsoft Corp’s Office Communications Server and Cisco Systems’ Unified Communications. In this brave new unified world, can the network still deliver the same consistent, day-in-day-out quality of experience? Will it? And if it doesn’t, what recourse do business customers have? Will multiple equipment vendors and service providers play the blame game, leaving customers in the lurch?Certainly, the quality of experience can degrade on IP networks, as most telecom and IT professionals know by now – if not from personal experience then from scare stories. IP voice, or video, packets can be lost or arrive late, causing jitter or noticeable latency and resulting in audio anomalies such as clicks, slurring, clipping, or awkward pauses that turn a telephone into a walky-talky. Echo and distortion can be a problem. In video, the same root causes result in pixilated images, jerky motion, poor synchronization of voice and video. How frequently they happen depends on a great divide: just as in the old days, there is still a demarcation point between the enterprise and the carrier. When assessing a situation you’ve got to know which side you’re on.If you’re starting to implement a unified communications strategy by deploying on-premise IP telephony and going out to the PSTN through gateways, the news is generally good. But once you move past the demark and contract with a service provider to carry IP voice or video traffic for long distance calling, or over private network links to branch offices or to a managed PBX, the picture gets considerably murkier. The Sudbury & District Health Unit: lessons from a pioneerNot that poor quality of experience is a given, far from it – witness the case of the Sudbury & District Health Unit (SDHU). The SDHU is an IP telephony pioneer that we’ve written about before. It switched to a Cisco-based IP telephony system in 2000. Today, it has a main site, four branch offices connected by private IP voice lines and 350 phones in total. “We didn’t have any problems with quality of service, never have, and there’s a reason why we haven’t,” says Craig Tyers, senior network analyst at SDHU.The reason? SDHU extensively overhauled its network infrastructure to accommodate voice at the time it switched to the new telephone system. “That’s one of the biggest recommendations I make – do both at the same time,” Tyers says.The new infrastructure did the following: it delivered adequate bandwidth and redundancy; it included power over Ethernet (PoE) to get power to phones; and it supported quality of service (QoS) functionality. The last of these can make all the difference, as it allows the network to tag voice packets and give them higher priority than computer data packets, so they’re much less likely to be delayed by other traffic on the link. At SDHU it wasn’t very difficult to configure QoS, either. Even with the early Cisco IP telephony gear the health unit started with, it was a simple matter of telling the routers and switches that phones were present, and they automatically did basic QoS configuration, which worked fine. “It was like, flick the switch and up it comes – and we didn’t have any issues,” Tyers says. QoS technology has been refined to the point where it’s pretty much a non-issue, says Jayanthe Angl, a senior research analyst at London, ON-based Info-Tech Research Group. But Angl adds, “Maybe non-issue is not the right word. QoS is still fundamental. But the best practices are well defined. And it’s rarer now to have very basic issues with QoS.”Info-Tech is in the process of surveying Canadian companies about their plans and concerns around unified communications. Ensuring QoS is not a big concern that has come up, he says. Veteran telecom consultant Marty Parker, a partner at California-based UniComm Consulting LLC, generally agrees that QoS on the LAN side “should be a very infrequent problem.” However, QoS technology is not infallible, he cautions. When big data packets can get you downFor example, in environments where very large data files travel over the same network links using protocols such as FTP, QoS may break down because data packets are so large and take so long to travel over the link – temporarily plugging it – that even with tagging and prioritization, some voice packets may be delayed or lost.The same thing is true if there simply isn’t enough bandwidth in the link to support the volume of traffic being pumped over it – although this is more likely a problem on the wide area side. “If the link is congested,” Parker notes, “there’s not much QoS can do.”There are solutions to both problems. Creating virtual LANs with bandwidth dedicated to voice, which is possible with most modern network infrastructure, solves the problem of over-size data packets delaying voice packets. Simply increasing the bandwidth in a link, over provisioning, solves congestion problems.Configuring QoS and following other best practices around implementing IP telephony may not always be as simple as it was for SDHU, though. “It requires a fair amount of expertise,” says Parker. “But at least everybody now knows what it’s going to take – that you have to do a network assessment before you start, that you may have to re-engineer the network.” Indeed, network upgrades often cost more than the IP telephony system, he says. “Which is why Cisco loves this application!”The latency and jitter problems that are common in the absence of properly configured QoS are not the only quality of experience issues. Echo is sometimes a problem. This is often mistaken for a QoS issue, Tyers says. In fact, it usually has more to do with tuning gateways properly at the point when the IP telephony calls go out on to carrier PRIs (Primary Rate Interfaces).Apparent noise on the line and audio distortion can be problems, but they’re often caused by users turning the volume up too loud on phones. “The technology is somewhat sensitive to cranking your volume to the max,” Tyers notes.SDHU has also had excellent success with wide-area links to its branch offices. It mainly uses fibre-based services from Agilis Networks, a subsidiary of Greater Sudbury Utilities Inc. “I have not had a problem with my municipal area network ,” Tyers says. “They’ve been great. I do not have any reports of any voice quality issues. Period.”But that may be mainly a function of the relatively short distances covered, the amount of bandwidth SDHU pays for – and choosing the right carrier. “As you go outside your building, you have no control over quality of service,” Tyers contends. “I mean, the simple reality is, you’re trusting the provider to accept what you’re giving him .”The SLA is your (fair-weather) friendTyers has an SLA with Agilis, and it offers some assurance that voice quality will be maintained. But the real problems on the wide area side occur in more challenging scenarios than SDHU experiences.What happens, for example, to QoS prioritization when IP voice traffic hits a carrier’s long-haul shared network? As Tyers notes, any one of the links, routers or switches that voice packets pass over or through on their way to a final destination could cause problems. Carriers now offer services using MPLS (Multi Protocol Label Switching) that theoretically preserves IP PBX QoS tagging from end to end. And there are other network technologies that can provide assurances of good quality, including Resource ReSerVation Protocol (RSVP), which in effect reserves adequate bandwidth over a link for a given connection.But ensuring end-to-end quality still depends in part on carriers being able to monitor traffic not just on their own networks, but on the customer’s network too. Using products from companies such as Psytechnics Limited and Ditech Networks, carriers can monitor traffic in real time and use data mining techniques to diagnose problems. However, unless the customer deploys the same tools, or allows the carrier to deploy them to gather data, end-to-end monitoring is impossible. And some customers are loath to give carriers access to their networks. It’s sometimes the carrier’s network that is at fault, of course. But if the service provider can’t do real-time traffic monitoring from end to end, it can’t or won’t guarantee QoS, says analyst and consultant Lisa Pierce, a vice president with Forrester Research Inc. Pierce believes carriers should try harder to convince customers to allow access. “Carriers could say, for example, ‘If you let me deploy these tools to rapidly diagnose problems, we can give you faster time-to-repair .’ But they aren’t thinking that way. If the time to repair were going to be meaningfully shorter, I think a lot of companies would be willing to .”“Right now, ‘Pay me and trust me.’ Hello! What planet are you from?”Carriers targeting enterprises certainly offer SLAs on IP voice links, and they do include provisions for measuring latency and packet loss, two of the key metrics determining QoS for IP voice. But these agreements are inadequate for a couple of reasons, Pierce says. First, the window of measurement is too large. They measure average performance, typically, over a month. It only takes a few minutes of bad QoS performance to drastically reduce quality of experience. And almost any change to the network – even something as simple as changing security settings – can “wreak havoc” with QoS on a temporary basis, she points out.One client of Pierce’s, a company running voice calls over MPLS links to multiple contact centres, found this out the hard way. “There had been a number of small outages,” she explains. “The carrier didn’t care because it was still able to meet the terms of the SLA. But it wrecked the customer service experience .” Carrier SLAs also typically measure what happens with simulated voice traffic, not what really happens, which again may skew results and disguise real-world problems. Worst of all, for many customers, SLAs are not open to negotiation. “For small- and medium-size companies, realistically, there’s not a lot of flexibility for demanding better quality of service,” Angl says. But for customers with very large traffic volumes, deep pockets and a lot of clout, SLAs are negotiable, Pierce says. She has a client that put one service provider on notice recently that monthly average measurements are no longer acceptable. The client is demanding measurements be averaged on a daily basis. “The is in shock,” she says. “But they have to get there.”Pierce believes customers need to dig their heels in on the issue of SLAs, and possibly band together as they once did with pre-deregulation to gain power over carriers. Too many carriers, she charges, work on the assumption that it’s inevitable IP voice will be poor quality compared to legacy voice, and that customers will accept it. They shouldn’t accept it, she says. Nor is it inevitable. “But have to really want to work at it.”Video ups the anteThe increasing use of video, meanwhile, both for conferencing and corporate narrow casting, complicates the business of maintaining quality of experience. “The same issues apply,” Parker says. Only in spades. Video of course demands even more bandwidth on the network. SDHU is now doing video conferencing over the managed IP network operated by the Smart Systems for Health Agency (SSHA), an Ontario government agency providing services and shared infrastructure for health care organizations. It has worked “flawlessly,” Tyers says. It is also using Acrobat Connect Professional (formerly Macromedia Breeze) from Adobe Systems Inc., which does not work flawlessly. The reason: video travels over the often congested public Internet. More and more businesses, especially small companies, may be growing tolerant of low-cost Internet-based communications services such as Skype that are “good enough.” But the minute you leave the managed network, all bets are off on QoS. Conversely, if you stay on a managed network and provision enough bandwidth, video – and voice – QoS is a relative no-brainer. As Parker notes, eye-popping demonstrations of HD video conferencing systems such as Hewlett-Packard’s Halo and Cisco’s TelePresence work as well as they do in large part because the links are provisioned with loads of bandwidth – in some cases multiple OC3s (156 megabits per second).On the other hand, unified communications modes such as instant messaging, presence, even Web conferencing, require relatively little bandwidth and because they’re non-simultaneous, are affected very little QoS issues. In fact, they may be the eventual antidote to QoS concerns in unified communications. “Communication is shifting,” Parker says. “We’re seeing more collaborative work spaces and people using instant messaging. We’re moving away from some of the challenges of voice quality. A lot of engineering has gone into VoIP. But voice is no longer the dominant mode of communication. E-mail is already more important, and IM and social networking are becoming more important. They’re replacing the voice conversation.”All very promising, but when your customers are complaining about jitter and latency on IP voice calls today, it’s small consolation.

Don’t settle for poor quality of service for IP voice: demand better from your carrier

News | 06/14/2008 5:28 pm EDT

To keep reading, get a free trial.

Already a Subscriber?

Reuse & Permissions

Inside The Wire Report

Information

Hill Times Publishing

Get in Touch With Us