Who Owns Application Performance?

Too often, no one – and that’s not good enough Who’s in charge of application performance in your company? Peter Sevcik, president of NetForecast, argues it’s about time enterprises considered creating a role specifically for this crucial measurement. I visited several enterprises recently and was struck by the abysmal state of application performance management. Let me describe two companies. The first is an international manufacturer where I visited the network group to discuss the performance of a mission-critical application. In fact, the folks in the network group introduced me to the application users in the office. The reason I was at this office is because the application users had been complaining about response-time performance. I always ask a lot of questions about the application, the network path to the servers, how they are accessed and what the users actually do with the application on a daily basis. The network manager replied with a single sentence of about 10 words and then said, "And that is all I know about the application, period." We then had to wait for a complex diplomatic process within the organization to have the network folks introduce me to the users. It turns out the network folks had actually never talked to these users before, even though the users were only two cubicle rows away from the network group. The second company is a serious research organization with thousands of employees scattered around the globe. In this case, I talked to an application manager who knew, based upon user reports and personal observation, that some of the users saw very poor performance. When I explained some of the performance acceleration technologies that may help the situation, she said, "Why did my network staff not tell me about these options?" In fact, the network group had constantly reported great network performance and simply washed their hands of the problem. What do these two stories tell us? I am sure you have heard the saying, "I work in a mushroom factory; they keep me in the dark and feed me shit." I bet the network managers in both of these companies repeat that line from time to time. Here is my reply: "You like to hide in your dark corner and don’t want to be bothered with what is really going on over your network." This lowest common-denominator approach to application management is called silo management. Silo ManagementWhen an enterprise buys a new technology, the firm often purchases an element manager to make the system work. Over time, the element managers are grouped so they align with the organizational structure of the company. The result, shown in the accompanying graphic on page 16, becomes the default enterprise management strategy. Enterprises will work hard on defining strategies, policies, procedures, training, help desk and even SLAs for each silo so the systems work like finely oiled machines, but separation leads to trouble. There are often many silos. For example, the single WAN silo can easily become separate silos for the private network, IP-VPN service, and the public Internet. I recently saw a big bank operating different management systems for Web, mid-range and mainframe servers. These basic management investments are needed to diagnose and repair broken parts of any single silo, given that you can find the break. And therein lies problem number one: which silo is broken? So some enterprises go farther to integrate the information from each silo into a manager of managers, business analytics engines, event aggregation systems (alarms), event correlation (fewer alarms), automated threshold algorithms (just the right number of alarms) and, of course, a huge repository of data, most of which is never read. Many vendors work hard at trying to improve this state of affairs by adding end-to-end or application-centric views of the data gathered by each silo. The thinking goes that if you can just get all the data from all the silos into one place and correlate it by some means (typically by time), then you will see an unusual pattern that indicates an anomaly. Why Silo Management FailsNone of the extensions and improvements to the silo data actually gets to the heart of the issue, which is to figure out why a real end-user is experiencing a problem. In fact, the added cleverness is often fuel for the assertion, "There is no problem in my silo." The person who just made that remark might add the following under his breath: "Now leave me alone so I can go back to my dark corner." One would think senior management would not stand for such an attitude. But they do, and here’s why: management thinks it understands how the system works. You wouldn’t be a manager unless you convinced yourself and your boss of that. They "know" how user tasks operate inside the silos and are then passed from silo to silo. They think the reports they are reading are reflections of the view into each silo, which informs the logical – if inaccurate – path a user takes as he moves through each silo on his various tasks. If the path is assumed accurate and the user’s steps through it assumed accurate, all must be well and the complaining user must be hallucinating. But the humans watching all the silos really do not know the path of a user task. The path changes over time as systems change and are upgraded. The user does things that were not foreseen. Complete subsystems move to new locations. Security and acceleration devices add proxies, filters and even change how the application operates. The result is a much more complicated user task-path – accurate, but difficult to follow. The path criss-crosses some silos many times. There is no way to see and understand the real behaviour of a user task without following the task explicitly. You can’t find an illegal activity by asking each bank to tell you the total funds on deposit each day. You must follow the money account by account and transaction by transaction. I am not proposing that enterprises spend the huge resources required to track the details of every user task. Instead they need to watch the resulting application user-response time as the key indicator of overall application performance. If the task response time starts to degrade, then detailed investigations should be performed on the paths those tasks take. Despite all the talk about business oriented performance metrics for IT (dashboards, business logic, et cetera) and full end-to-end views of infrastructure (very sophisticated silo integration) few tools truly look at real-time user flows. Application Performance Manager NeededProper performance management is delivered in two parts: Ongoing end-user performance measurement, reporting and tracking, followed by…flow-based diagnostics to find and solve specific problems. But no group within an enterprise has this responsibility. In the real-life examples I mentioned earlier, no single person was to cut through the silos, following a real user’s task path. I recently found a small group within a bank that does have this job. They have excellent tools and methods for finding the real performance problem. They operate the second half of the performance management strategy: flow-based diagnosis. The first half is performed by a business manager listening to his users (no formal measurement and reporting exists) and then going to each silo manager several times with no resolution. After hearing "not in my silo" often enough, he hires the in-house team of cross-silo experts to really figure out the problem. Enterprises must move from these ad hoc approaches to a formal application performance management process that is owned and operated by a formal group within the company. We won’t really address application performance until we put someone in charge of the problem and give him or her the resources required to get the job done. This will take more than funding. It will also require new relationships among groups within a company, and new management procedures. Proper performance will be realized when a company can point to a person in charge and read the ongoing reports from that person documenting real end-user performance. In addition, employees who hide behind a narrow job description and show no initiative to fix problems should be re-educated or shown the door. Companies should not let anyone become a self-made mushroom. Anything less is insufficient for a competent and competitive enterprise. This article is reprinted by permission of Business Communications Review (www.bcr.com.)

Who Owns Application Performance?

News | 09/01/2006 4:00 am EDT

To keep reading, get a free trial.

Already a Subscriber?

Reuse & Permissions

Inside The Wire Report

Information

Hill Times Publishing

Get in Touch With Us