We've said it many times before, and we're not shy about repeating ourselves: System administration is the most important support function for Unix distributed-computing environments. The two most important aspects of how we approach system administration at Sun are the partnerships we develop with business units and users and the automated systems we use for fast and efficient delivery of services.
In previous columns, we described those partnerships and how, by formalizing the process with our Unix Production Acceptance ("Our #1 priority," June 1994) and Service Level Support Agreement ("SysAdmin: The heart," December 1994), we make sure Sun's distributed-computing systems best serve the company's needs. This month, we take a look at the protocols and automated processes we use to respond to requests for support.
Are you being served?
The help desk -- the centralized system that responds to ad hoc user requests for system administration services -- has become a nearly permanent fixture in most medium to large organizations' computing infrastructures. Like most help desks, ours at Sun was originally based on the most traditional and ubiquitous way users could and would call for help: the telephone. A help desk administrator would either resolve a user's problem then and there over the phone, or take down and relay the necessary information for later system administration response. Telephone-based help desks do work, but they're extremely labor intensive, inefficient, and expensive. Under pressure to reduce our budget (five percent per year!), we did what any red-blooded technocrat would do: automate. We created ServiceDesk, a graphical-user-interface-based on-line application that lets users report their systems problems and request help from system administration electronically over Sun's network (see screen shot of ServiceDesk). ServiceDesk's suite of tools also tracks user problems and resolution.
With ServiceDesk, users prepare their own "trouble tickets" and work request forms, which are then automatically submitted over Sun's network to the appropriate system administration group. That cuts the time help desk administrators spend taking user requests for assistance and directing the call to the appropriate person or group for resolution. Moreover, ServiceDesk tracks trouble tickets and work requests all the way through to resolution, including notifying users of their progress, so few, if any, problems "fall through the cracks." In fact, ServiceDesk will automatically escalate user requests to management's attentions if they haven't been resolved within a specified period of time. Now that's responsiveness!
Help, don't hinder
All of our rightsizing efforts to "Unixfy" Sun's systems have a single motto: SELL the concept; DO NOT shove it down users' throats. Work with your user/customer or a rebellion will surely happen. Regardless of the benefits and efficiencies from ServiceDesk, we still had to sell it to users. Think about it -- picking up a phone is easy, and personalized service is usually preferred by customers. With ServiceDesk, requesting help means filling out a form and sending it through a machine to "who knows where."
As expected, there was some yelping when we first went live with ServiceDesk. True to our "Unixfied" approach to innovation (collaboration versus confrontation), we continued to run the traditional phone-in help desk. Whenever a user would phone in a request, however, we would remind them (politely, of course) to use ServiceDesk. Over time, we were able to show users that our response times to resolve problems improved dramatically through ServiceDesk over the traditional phone-based help desk: On average, the help desk took nearly four hours to resolve problems. Not long after we installed ServiceDesk, our average turnaround time for help was reduced by half. We now promise to respond to user trouble tickets within 30 minutes (the help desk used to take two hours) and guarantee resolution within four hours or else the problem is brought to management's attention. (Just a note: In the nearly five years we've been with Sun, no systems problem ever escalated to management's attention. Whew!) System administration also promises to resolve work requests -- submitted by users through ServiceDesk when they want new systems installed, existing systems upgraded, or their operations moved -- within five business days.
There's one other important aspect to ServiceDesk that truly makes it work: user feedback. ServiceDesk automatically keeps users informed of the status of their requests, including notifying them by e-mail when the task is completed. And ServiceDesk automatically sends users a Feedback Tool (see inset to ServiceDesk screen shot) that lets them quickly and easily rate the quality and timeliness of the service we've performed. We assure them that all their smiley-, neutral-, and frowny-face feedback is given serious consideration -- our system administration performance evaluations by management depend on it. We're allowed no more than two percent frowny faces, and every unsatisfied user gets a personal phone call from someone in system administration. Fortunately, the grand majority of our respondents are all smiles.
Our goal is to eliminate the help desk altogether. We haven't done that yet, but we have made significant progress by streamlining as many of those old MIS help desk functions as possible. For instance, at one time there was a lot of "passing the hot potato" from the Assistance Center (nŽe help desk) to a centralized "focal" queue of system administrators, and perhaps further along to an on-site or near-site system administrator to resolve tough trouble tickets.
Today, ServiceDesk trouble tickets and work requests go directly to a focal queue of system administrators assigned to one of Sun's campuses or business-unit territories for resolution. And our focal queue system administrators and the on-site or near-site administrators, who each have a specialty such as database or systems management, work as a team to solve all problems.
A recipe for support
Here are the steps Sun's system administration staff now takes to resolve trouble tickets:
Work requests for systems installation, updates, or relocation (work requests) are handled a bit differently than trouble tickets:
Harris Kern (firstname.lastname@example.org) is Sun's Open Systems Migration Consultant for NAAFO Market Development. Randy Johnson (email@example.com) owns R&H Associates, a full-time rightsizing consultancy in Boulder Creek, CA. R&H Associates helps people worldwide in implementing and supporting client/server infrastructures based on their proven methodologies. © 1995 Harris Kern and Randy Johnson. All rights reserved.
Pick up a copy of their book Rightsizing The New Enterprise: The Proof Not the Hype, SunSoft Press/PTR Prentice Hall, ISBN 0-13-132184-6, or their new book Managing The New Enterprise: The Proof Not the Hype by Kern, Johnson, Hawkins, Law, and Kennedy, SunSoft Press/PTR Prentice Hall, ISBN 0-13-231184-4. Browse SunSoft Press offerings at: http://www.sun.com/smi/ssoftpress
You can buy Managing The New Enterprise and Rightsizing The New Enterprise at Amazon.com Books.
If you have problems with this magazine, contact firstname.lastname@example.org
Last updated: 1 January 1995.