12.19.07

How to measure usability? The way your users do, of course.

Posted in Usability and Design at 4:39 pm by justindz

Time to drag out the old usability soap box. It’s starting to bend in the middle, and so I might need to buy another year’s supply of bulk soap to get a replacement.

I spent some time messing around, as a nerd/technologist and child of the Internet revolution, exploring the virtual world of Second Life last year. I will spare you a re-creation of my assessment. Suffice to say, it’s a great concept and has some interesting value for proof of concept or collaboration work but is hampered by poor usability in the client GUI as well as poor performance usability on machines with less power than something like a Cray or perhaps the Amazon or Google computing clouds. Well, not that bad, but I could definitely hear my Mac Mini struggling to vent heat while my avatar was “walking around” the Adidas virtual island.

An interesting article came across the Reuters Second Life bureau RSS feed this morning reporting on claimed versus perceived performance improvements in Second Life. Linden Labs, the company behind Second Life, made the claim that performance had improved using the metrics of server-side frames-per-second (FPS), a measurement of how fluid the 3D images are being presented to users, and the occurrence of crashes. A user poll, however, indicated that users were perceiving a simultaneous steady decrease in performance. The issue, client-side FPS was creeping downwards despite gains in the server-side measurements.

The lesson here is one that I’ve often repeated within GXS. The only way to measure usability *really* is to measure it from the user’s perspective. We have a re-design to one of our Trading Grid Messaging Service dashboards, for example, which moves one data element to an asynchronous lookup so that the dashboard can draw very quickly and the user can review 9 out of 10 metrics while the 10th is populating. Performance usability improvements often focus on optimizing data display so that (for the sake of the argument) a 10 second page render moves to 8 or 9 seconds, on average, when you can make significant usability improvements by moving to a 3 or 4 second average page render and defer a single data element to asynchronous. The user needs time to review the other data elements anyway, so the overall impact is positive. In Linden Lab’s case, the server can push out content faster and faster and faster, but if client performance doesn’t change then nothing has improved in a meaningful way for the end user.

On the upshot, it’s clear that Linden Labs is doing the right thing by being sensitive to this issue and responding to it, rather than continuing to tout their purely provider-side enhancements.  I was going to dovetail into applying this principle into airline customer satisfaction during the holiday season, but you probably don’t want to read this entry for another few hours. Suffice to say, the carrier with the least over-booked and canceled flights shouldn’t have reason to brag. Whether you travel or stay home this season, make sure you have a good time :-)

Leave a Comment

You must be logged in to post a comment.