05.24.07

I recently attended the Forrester IT Forum at the Gaylord Opryland in Nashville. The conference, and the hotel are both favorites of mine, though sadly I experienced less of each than I would’ve liked. Having said that though, the sessions I did attend were excellent.

Some of the best sessions are often surprises, and for me that was the case with “Using a Goal Tree to Measure and Manage”, by Jean-Pierre Garboni (VP of Forrester). I was a quantitative business major in school (no, that is not an oxymoron, it is called management science or operations research), so I thought it might be of interest. Far from an esoteric discussion of quantitative methodology though, it was asystematic attempt to explain why the millions of dollars of investment in monitoring have not given us more insight into customer experience.

The answer lies is a pattern so common it is almost a cliche — we are measuring and monitoring infrastructure one piece at a time, and thereby failing to get the “big picture” (I am really, really over simplifying here…). Basically, we are best at measuring things like uptime, transaction rates, queue depths, etc; all of which contribute to outcomes, but are not good representations of the entire outcome(s). This is further complicated by the fact that there are different outcomes for operations,”the business”, and the customers.

This may seem obvious, but if you are currently building/using SOA based applications and services, I’ll bet you’re not ready for this. For instance, if you have the ability to dynamically add capacity to your SOA service, is that load management capability talking to you operations management infrastructure? If I add a node and it runs slow, can I recognize the solution that is impacted? Can I resolve down to customers? This has always been an issue, but in an area of SOA and virtualization,it is an even bigger deal.

The exciting part of this talk, from my point of view, was the methodology. Basically, you start with a high level goal (roughly equivalent to the successful outcome), and a modeler (person, not a piece of software) decomposes that into systems, processes and ultimately functions with measurable results. The idea is that the functions are well-suited to the current suites of monitoring/measurement, and the model helps us to roll this up into a fair approximation of the outcomes. The largestbenefit may in fact be the way this approach gets people thinking ‘top-down”.