Originally published in the October 1994 issue of Advanced Systems.

Unix Enterprise

Managing integrated systems

We had to make our own tools to glue together Sun's distributed IT.

By Randy Johnson & Harris Kern

As we embarked upon our rightsizing efforts at Sun, you can imagine the scare we took when we discovered there were few Unix systems-management tools around -- a very scary situation for us RAS (reliability, availability, serviceability) disciplined mainframers, you can be sure. We had to build our own.

Today, there are many fine Unix management products to choose from, both from systems vendors and third-party developers. In this and future columns, we'll describe the collection Sun IT has built over the years, so you get a sense of what's needed and what you might look for when evaluating management products for your own New Unix Enterprise.

Paperless Reporter
A principle role of IT is what we call doing the Four Rs: distributing the Right Information to the Right People at the Right Time for the Right Price. We've argued in the past that the best way to accomplish the Four Rs is through a distributed-computing environment, because it's the best, sometimes only, infrastructure for widespread delivery of electronic information. But a real problem is collating, translating, and delivering reports that come from heterogeneous, widespread sources to the variety of people that need them. It's not surprising that many organizations still print out and deliver critical information on paper -- reams of paper -- to the various individuals and groups that need to read them and respond.

Moreover, because the systems don't allow it or because it isn't easy to do so otherwise, entire reports and documents are printed and distributed, even though users often need to see and work with only a small portion. That leads to a lot of waste!

One of our first integration-management tools helped solve that high-volume print problem, and it solved many other report-distribution issues, as well. Paperless Reporter is a general-purpose, integrated distribution and viewing system for on-line reports that originate from any of Sun's systems, including Unix servers, mainframes, and minis. A centrally maintained database manages report collection and distribution to a secure subscription list throughout the organization, along with information about each report (format, and so on).

Paperless Reporter delivers the documents electronically over the network to private-user directories on local servers. Users can then read the reports through it's viewing tool or with other tools, like their favorite word processor or spreadsheet. They can still get a printout, but with Paperless Reporter, that usually means printing selected portions of a report on a local printer -- much less paper and fewer distribution problems than before.

We sold this integrated, distributed reporting system and its goal of saving time, effort, and paper to our user community by giving those who traditionally received hardcopy printouts a T-shirt that said, "Save a Tree. Use Viewmaster" (Paperless Reporter's name before it became a commercial product). It worked. Over the last few years, Sun has saved millions of pages of paper and was presented an Ecology Award from Santa Clara County in California for saving trees -- not to mention the very real improvement in people's productivity.

Software distribution and version control can be the most virulent sources of IT headaches and debilitating chaos for a distributed-computing enterprise -- if you don't apply centralized controls. SunDans (Distributed Administration of Network Software) is a set of standards, mechanisms, and system administration practices that provide a transparent means of delivering new and updated software to users over a network. Users' local servers subscribe to individual and suites of software; SunDans automatically distributes and installs the products over the network. SunDans also can manage floating licenses so large groups can share the cost of occasionally used software.

SunDans was one of the first tools we ever developed, and it has paid off: Sun IT decreased the ratio of system administrators to users by more than 20 percent. Much of those savings come from reducing the need for on-site software management and support. Certainly this tool was a key part of our supporting distributed environments.

Bridging legacy to Unix distributed systems is a must for most rightsizing organizations. SunRAI (Remote Application Interface) does just that: It provides a framework and Unix API that enables programmers to construct pathways (transactions) from existing legacy applications to Unix-based software without modification to critical mainframe systems. It allows batch and real-time transfers and it maintains existing application security. SunRAI also guarantees that intersystem transactions are transmitted securely and in proper, prioritized order. It maintains recovery facilities against network or SunRAI component failure. SunRAI keeps a complete log of messages so administrators can generate status reports, tune the system, and resolve problems.

SunRAI is critical to us for satisfying many of our business requirements. For instance, through SunRAI we are able to automatically communicate manufacturing orders (up to 2,000 transactions per hour) from our European-based Unix MFG/PRO system to our mainframe sales-order processing system in the U.S., where Sun's manufacturing processes still reside. Before we developed SunRAI, users in Europe had to separately type in their sales orders into their Unix systems and then turn around and re-enter the same order into our mainframe system -- a process not only inefficient, but prone to errors. Now, one order entry automatically appears on both systems.

Console Server
Distributed systems mean lots of consoles all over the place, often so far-flung geographically that it would take an army of system administrators and divisions of technical support to manage them all. Rather, Sun IT developed a set of tools and processes called Console Server that replaces many of those individual system consoles with a single desktop. It lets technical-support staff (systems programmers and administrators) control systems from remote locations.

At Sun, an on-call technical-support person is automatically paged when a system or application problem occurs on any server that contains a mission-critical production application. From home or office, that person can then remotely login to a Console Server, analyze and sometimes fix the problem, and even reboot the production server remotely. Often there's no need for the administrator to even drive into work, let alone visit the server room where the problem occurred. That way, most problems can be quickly resolved without a lot of bother and overhead -- the essence of high availability. Console Server also keeps a history of all console messages from each system it serves, rather than just the limited screenfull or two that appear on a traditional ASCII-terminal console, so administrators can better diagnose and fix chronic as well as acute problems.

Scripts, too
Distributed systems work best when molded to the requirements of the business unit or department they serve. So, too, should administrative tasks fit the environment. Sun's Data Center has an ongoing program of custom-tool development, much of which is programmed as Unix shell scripts. Like CLISTS for mainframes, Unix shell scripts are collections, mostly in ASCII, of commands and programming logic for automated, unattended operations of Unix-based systems. Everyone on our staff is involved in shell-script development, from computer-operations personnel to senior technical-support folk, because they recognize the benefits provided by these scripts and realized immediate productivity gains through automation of tasks usually performed manually.

We have bundled many of these internally developed shell scripts into what we call the Sun Data Center Scripts. They cover three areas:

  • Availability is our #1 our goal. The very first, production-grade script we built was called the Morning Report -- and we use it to this day. It automatically polls and reports server availability on a daily basis so we can measure our #1 goal and anticipate problems.
  • System Administration is a collection of scripts that automate tape backup, disk usage, peripheral installation, and preventative maintenance. The preventative maintenance script is similar to the MVS mainframe error-reporting utility EERP. It reports "soft" errors on hardware so we can schedule downtime to replace the defective part before it becomes a real problem.
  • Sybase and Oracle Administration is a standard set of database utilities like those for mainframe database-management systems. The scripts automate database creation and installation, import and export databases, check for errors and consistency, dump and restore transactions and whole databases, and monitor capacity and operating statistics.
  • Get glued
    As we said earlier, we built these and many other systems-management tools and utilities because we had to. Since we started rightsizing Sun, many others have seen the need and developed their own management products for Unix distributed systems. Many that we developed internally, as well as the many third-party products, are now commercially available. Go out there and get some of that glue for your organization. []

    [Harris & Randy photo] Harris Kern (harris.kern@sunworld.com) is Sun's Open Systems Migration Consultant for NAAFO Market Development. Randy Johnson (randy.johnson@sunworld.com) owns R&H Associates, a full-time rightsizing consultancy in Boulder Creek, CA. R&H Associates helps people worldwide in implementing and supporting client/server infrastructures based on their proven methodologies. © 1994 Harris Kern and Randy Johnson. All rights reserved.

    Pick up a copy of their book Rightsizing The New Enterprise: The Proof Not the Hype, SunSoft Press/PTR Prentice Hall, ISBN 0-13-132184-6, or their new book Managing The New Enterprise: The Proof Not the Hype by Kern, Johnson, Hawkins, Law, and Kennedy, SunSoft Press/PTR Prentice Hall, ISBN 0-13-231184-4. Browse SunSoft Press offerings at: http://www.sun.com/smi/ssoftpress

    [Amazon.com Books] You can buy Managing The New Enterprise and Rightsizing The New Enterprise at Amazon.com Books.

    [Copyright 1995 Web Publishing Inc.]

    If you have problems with this magazine, contact webmaster@sunworld.com
    URL: http://www.sunworld.com/asm-10-1994/asm-10-unix.html.
    Last updated: 1 October 1995.