![[Copyright 1995 Web Publishing Inc.]](/sunworldonline/icons/b-copyright95.gif){kind=icon}
The lure of near-limitless storage is certain to catch the eye of Unix system administrators faced with the prospect of handling a growing mountain of data. Like wealth, disk space is something you can never have too much of, and HSM (hierarchical storage management) offers a way to coax gigabytes of storage out of megabytes of magnetic disks.
HSM sounds like a great idea. Rather than spend lots of money on expensive magnetic disks, you spend a moderate sum on optical jukeboxes, platters, and software. Then you set up your migration partitions and collect data till the cows come home. HSM software (usually) stitches into the kernel and catches calls to files in migration partitions. Files get migrated based on simple criteria, such as access or modification times or size. When a migrated file gets accessed, it is recalled from jukebox storage and made available just as if it were originally on magnetic disk. Cagey HSM packages keep a portion of the file on magnetic disk, so simple accesses, as with file, can be satisfied without dredging the migrated file out of a jukebox.
HSM is tricky to implement. It's not easy to fool a system into thinking its data is elsewhere and then retrieve that data reliably. One stumbling block to perfect HSM is backup. Commonly, files are migrated out based on access time (files least recently used are punted to an optical juke). Ordinary backup utilities cruise down a filesystem copying files to tape. It would be disastrous if this process resulted in every migrated file being hauled in from a jukebox, and worse yet, if the migrated file were entirely off-line, in a tape vault, say. To dance around this problem, HSM vendors often provide a means by which the migration/retrieve can be sidestepped. This can be accomplished with a special group whose processes see stub files for what they are, rather than the data mirages the rest of the system sees, or by a wrapper routine that swathes programs and prevents migrated files from being retrieved.
The packages we reviewed have a means by which filesystems are spread over multiple optical platters as part of their feature set. In some cases, as with products from Alphatronix and Emass, these platters can be mounted as separate Unix filesystems and accessed in a conventional manner without going through the overarching HSM software. Alphatronix and Emass, in fact, sell a separate optical filesystem product, which their HSM offerings use to read and write migrated data. ASC, OpenVision, and QStar embed this functionality within their HSM products.
Evaluating five HSM solutions was no easy feat. Our impressions of the state of the art are mixed. Alphatronix, ASC, and Emass have devoted effort to creating quality GUIs with which their systems can be easily managed. We appreciated being able to configure and launch an HSM strategy from an intuitive interface. The product from OpenVision, though possessing a few GUI tools, relies on command-line directives for much of its operation, and QStar's is strictly command-line based. Both of these packages could benefit from having their output retooled to be more easily understood by human beings.
All five worked (eventually). We ran into problems with most of them, but once their kinks were ironed out, data flowed smoothly from server to jukebox and back again. We have our favorites (Alphatronix's Inspire and ASC's NetArchive) and recommend to potential customers that they conduct their own usability tests before signing a purchase order.
The horror, the horror
Suffice it to say, our experiences with HSM software were less than ideal.
Never in the history of the reviews operation here at Advanced Systems have we been struck with such a high degree of failure. During the course of this HSM review, six workstations (five SPARCstations and one HP 9000/710) were in use as hosts for software and jukeboxes. We had to re-install the operating system three times on one machine because its kernel got hosed by an install script when it tried to un-install software (granted, the OS was Sun's Solaris 2.3, so it's difficult to assign blame). This machine and two others spent much of their time playing with Schr¬odinger's cat and filling the continuum between up and down states.
If the software side of HSM didn't vex us to nightmare, its hardware certainly would. Seven optical jukeboxes and one tape stacker were called into service for this review. Unfortunately, for various reasons, we used only the stacker and four of the jukes. The other three collected dust and all manner of oaths (details below).
All but one of the systems crashed at least once. The notable exception was a Sun SPARCstation 5 running SunOS 4.1.3 and hosting Emass's DataMgr software. Despite this sad record, at no time did we lose data, which is an issue of paramount importance to users.
ASC NetArchive 2.0
Advanced Software Concepts' NetArchive is the client/server champion of the bunch. A host of daemons resides on the storage servers, which communicate with attached storage devices. Clients run retrieve and migrate daemons that work with the storage server to access and migrate files. While this permits a network of machines to get into the HSM game, it has one limitation: Without the retrieve daemon running, users cannot access migrated files. For supported systems, this isn't a problem, but unsupported clients, such as PCs running PC-NFS, cannot play any HSM reindeer games. A migrated partition can be NFS-mounted from a storage server onto such a client, but without the retrieve daemon, its migrated data will remain inaccessible.
The NetArchive clients have a nifty little item we did not see in the other products (other vendors take note): a callback feature. When the storage vault is in use, the client software may not be able to open a connection with the server. When this occurs, the server passes back a callback key. This key is used by the client to register itself for the callback service. The client waits until the server completes its vault I/O and calls back. Once the connection occurs, the client performs the actions it originally intended. If the client does not get a callback from the server in 21Ú2 minutes, the client will initiate another call to the server.
An expansive GUI provided a window into NetArchive's configuration and administration, and after a lengthy period of pointing and clicking, we were migrating files to a dual-drive Pinnacle Micro jukebox from both a client and its server. That ASC's software could breathe new life into our godforsaken Pinnacle juke was joy enough; file migration was utter ecstasy. The GUI rivals that of Spectralogic's Alexandria backup product for sheer size (see "Taking a dip in the software backup pool," April 1994 for a review).
NetArchive mysteriously crashed once, though we couldn't reproduce the event. We offer a word of caution to NetArchive admins: Monitor its set of daemons occasionally to ensure they are up and running properly. Given this, it ranks high on our reliability list, sharing the firmament with Emass's DataMgr.
Alphatronix Inspire DMM 3.3
Our experiences with Alphatronix's Inspire DMM (Data Migration Manager) software were largely good. At no time did we lose data, though the machine hosting Inspire froze once, refusing even to respond to an L1-A sequence (the workstation's power switch served as a defibrillator). We suspect the cause of this mishap was a disk partition filling up as a result of a file being migrated back onto the system.
To help us test the software, we recruited an editor-in-chief to be a guinea pig and set him with two migrated partitions NFS-mounted to his local workstation. Surprisingly, he had no great complaint. He admitted to detecting delays of up to 30 seconds when accessing some files, but nothing on the order of what he expected. The lab was stunned. This individual (who shall remain nameless) is a system administrator's nightmare. We expected his usual obstreperous fits over Unix's quirks ("Why can't vi use a mouse!?"), not the docile HSM lamb that came bleating into the lab.
Configuring Inspire DMM and its jukebox-control component, Inspire, was so simple that we rarely had to consult its documentation. Elegant GUIs afforded us the necessary levers and pulleys to build an HSM solution exactly to our tastes. Unlike some of the other products in this review, Alphatronix's offering does not require a dedicated magnetic partition, nor is it confined to operate within a specified partition. Instead, we were free to migrate any directory anywhere on the server (though the software rightly refuses to migrate system files, such as /vmunix).
Its architecture is a layered design, with Inspire DMM riding on top of the Inspire jukebox stuff. Optical platters appear as mounted volumes and can be accessed as if they were magnetic disk partitions. Manually altering their contents is unwise, of course, as it may disturb the consistency between what appears to exist on magnetic disk and what actually resides on the optical volumes.
As we wrapped up this review, we learned Alphatronix is readying a new version of its software. The new version, which the company reports is based on object-oriented code entirely, is part of a suite of storage related applications.
Emass DataMgr 2.2
Three of the HSM offerings in this review sport well-crafted GUIs for command and control. Emass's DataMgr product joins those from ASC and Alphatronix in this league. The architecture of DataMgr is similar to that of ASC. It is built around the client/server model and consists of at least one server, one client, and one location daemon running on a network.
DataMgr is the HSM component that rides atop Amass, which is the optical filesystem and jukebox interface. Amass is required for DataMgr to operate. Being facile with Amass improves the use of DataMgr immensely. It provides full file, volume, drive, and jukebox management, as well as the tools needed to control, configure, and monitor the jukebox subsystem. Once we installed this partition of the software, we were on our way to migrating data.
Setting up migration partitions and configuring them to your tastes is a point-and-click pavane. DataMgr offers the usual list of party tricks: high and low watermarks, automatic and manual migrations, and GUI-assisted retrievals. However, DataMgr restricts migrations to a dedicated magnetic partition; you can't migrate directories outside this partition at leisure, as with Alphatronix's Inspire DMM, which is not restricted to dedicated partitions.
DataMgr sailed cleanly through our tests, foundering on no shoals and encountering no squalls. A particularly spiffy feature was its ability to suspend an I/O operation, say a copy to disk, while space is made available by migrating data to optical. DataMgr's command-line interface is strewn with options and joins OpenVision as a typing exercise.
OpenVision HSM 1.7
Like Inspire DMM from Alphatronix, OpenVision's HSM software works on top of a separate jukebox volume-manager component. Two administration tools and their array of GUIs handle the setup and configuration of migrated partitions, but a weighty collection of command-line utilities control the bulk of the migration, retrieval, and diagnostic functions. Once we learned how to change the background color of the GUIs from the ghastly default blue to something more tolerable, our appreciation for these tools grew. Though basic in design, the HSM GUI does automatically what would be tedious to do manually: edit the software's configuration files. And notable is OpenVision's on-line help, which is among the best we've seen.
With OpenVision we took a different approach. Tired of riding the technological leading edge, we configured its software to migrate files not to a slick optical jukebox, but to an Exabyte 10e tape stacker. This might seem hokey, but it worked and, surprisingly, it worked well. Sure, the access speeds of the 8mm tapes are slower than for magneto-optical platters (accessing one 11Ú2-megabyte file took almost three minutes with tape but only half a minute on optical, including load and seek time), but there is a corresponding difference in cost as well. Our humble 10e is less expensive than the modest 16-platter HP optical jukes we used in our tests, but it stores 50 gigabytes, considerably more than the HP's 20 gigabytes.
Other tape solutions, such as digital linear tape (DLT) or IBM 3490 formats, are more rugged than lightweight 8mm cartridges, but they come with weightier price tags as well.
Our main complaint about OpenVision's HSM software is the complexity engendered by the command-line utilities. To obtain a report of tape usage, for instance, you must issue a migdbrpt -a home command; labeling a tape requires something on the order of miglabel.sh home 000010 ct 1; and to consolidate the contents of several tapes onto one you're looking at migcons.sh home two ct 1 000009.ct.1 000008.ct.1 000007.ct. 1. A GUI encapsulating the functionality of these commands would be welcome and would avoid having to consult OpenVision's man pages or learn a dialect more akin to structured COBOL code than HSM software.
Once configured and running, however, OpenVision's offering became quite a data-storage workhorse. Initial anxieties over relying on tape were dispelled with use. OpenVision's HSM product works with optical jukeboxes, as well.
QStar MastarMind 2.2
MastarMind has no GUI, relying instead on a handful of commands issued from the Unix prompt. The bulk of these commands are accessible from /usr/local/etc, which is a nest of symbolic links pointing into the MastarMind install directory. QStar also symlinks /sys/QStar to point to the install directory, ensuring a common MastarMind path across all its supported platforms.
There isn't much to configuring MastarMind other than newfsing the dedicated magnetic partition that will serve as cache for the optical volumes, creating a set of platters (volume set) that will compose the optical filesystem, and then adjusting migration parameters to suit your needs. About half of MastarMind's commands work directly with jukeboxes; most of the other half work with the volume sets. All in all, it is a rather elegant assortment of utilities. Our biggest gripe is that the information contained in their output is often shrouded by numerical codes. Also, to our dismay, we learned that the version of the software we tested does not reclaim space on optical platters. We understand that a future release will possess this ability, but until then, rewritable magneto-optical jukes are made to behave like WORM drives -- an ignoble fate indeed.
For our tests, a single-drive HP jukebox served as the storage vault for MastarMind. Once we had created a volume set and copied data into it, we discovered that a single-drive juke was not the best choice. The system would often get locked into a seemingly endless succession of volume inserts and extracts (thrashing). QStar informs us that a multidrive unit would have fared much better. Sadly, our attempts to connect an Hitachi OL152 dual-drive model met with unmitigated disaster.
Late in the review, we switched from a SunOS 4.1.3 version of MastarMind to a beta version intended for Solaris 2.3 (which QStar says avoids the thrashing problem), but didn't have an opportunity to test the code fully.
When the system wasn't thrashing, MastarMind did as it should, migrating files from magnetic cache to optical disk and back. The software lacks some of the bells and whistles of other HSM contestants in this review, but in its simplicity is an elegance we appreciated after a long day slogging through the labyrinthine GUI of ASC's NetArchive.
Who designed this jukebox?
The ASTC was host to a total of seven jukeboxes during this review. Our experiences were wide-ranging and are best characterized by the ambient level of curses and salty epithets that drifted beyond the lab's walls. On good days, nary a peep was heard; on bad days, the nearby edit staff cringed at the vituperative torrent emanating from the ASTC.
The Hitachi two-drive OL152-60 model we tried to use with QStar's software was onerous to configure. To set the SCSI ID of the autochanger, you must open its front panel using the craftily hidden release lever. Behind the panel is a 3-D latticework of circuit boards, and on one of these are two slotted rotary switches. One sets the SCSI ID of the robot arm (goodness knows what the other one does). Changing the IDs of the two drives is no less painful. Microscopic jumpers slide onto a series of prongs on the back of each drive that set the bits for its SCSI ID. Needle-nose pliers furnished the necessary lilliputian touch required to jockey the jumpers.
Such diabolical fits of engineering were unnecessary with the trio of Hewlett-Packard model 20XT jukes we used in our tests, which were possessed of their manufacturer's usual legendary quality, and, whaddya know, ease of use, too. The SCSI IDs of the autochanger and sole drive are set through a four-button interface on each juke's front panel. These little Clydesdales worked like charms, giving us no grief whatsoever.
A Plasmon jukebox hung out with the dust bunnies in our lab for the duration of this review, since the software for which it was intended could not control its autochanger. Its gunmetal gray cabinet and stately presence were a welcome sight in the ASTC, though.
In an attempt to demonstrate cross-juke interoperability, we installed ASC's NetArchive on an HP 9000/710 attached to a dual-drive NKK Disc Inn (we're not kidding, that's its name). After a hardware fault and a visit from a service technician, we have nothing to report other than total failure; we never got the unit working with ASC's software.
Last and least on our list of jukeboxes is a Pinnacle Micro PMO 20-gigabyte dual-drive unit. A little more than a year ago we made the mistake of buying this jukebox along with Pinnacle's delightfully tacky yet unrefined virtual filesystem software. We never got the two working together reliably. After hours of tech support calls, we decided to request a refund and return the unit. With Klingon gall, Pinnacle refused this request, accused us of technical incompetence, and left us to our own devices. Our $19,000 investment languished unused until ASC's NetArchive product arrived. To our glee, ASC's software worked flawlessly with the Pinnacle juke, which was graced with HP optical drives (we understand that if it had been configured with Pinnacle drives, we would have had no better luck than with the unreliable Pinnacle software). Perhaps Pinnacle's software engineers should confer with ASC's and start writing code that works.
If we can offer any advice when shopping for jukes, it is these two words: Hewlett-Packard.
N is for Neville,
he died of ennui
Here is where we get to cough up some words of wisdom about our magical mystery tour through HSM software. This was a difficult experience for us. Prior to this review, three months of sensory deprivation testing Lotus Notes was the most arduous review we've ever attempted. HSM ranks first now.
All five of these products ultimately worked as advertised, with varying degrees of ease of use. High on the usability scale are Alphatronix's Inspire DMM, ASC's NetArchive, and Emass's DataMgr, owing mainly to their well-designed GUIs. OpenVision and QStar could bolster their products with nicer interfaces (or any interface at all) and be the better for it. Their creaky command-line approach and obfuscated output hail from a bygone era best left to archeologists.
Being easy to configure is good for system administrators, but what about users? ASC's daemon-mad client/server approach affords a high degree of data access, bringing migration and storage vaults to an entire network (assuming your clients are supported by its daemons). Alphatronix's Inspire DMM and Emass's DataMgr, with their clutch of daemons, purport to do pretty much the same thing, though we did not test this due to lack of time. OpenVision's and QStar's products are more closely tied to a single server, destined to migrate data solely within its confines. (We regret being unable to test QStar's NAP product, which provides connectivity between its MastarMind HSM software and a host of clients: DecNet, Novell, Windows NT, or Macs.)
Alaska's Denali Highway is a 120-mile stretch of unpaved optimism connecting two points that are pretty much in the middle of nowhere. If you were to travel this road, at first, you'd think it were pretty interesting. The scenery is magnificent, with Paleolithic glaciers on one side and caribou herds on the other; there isn't much traffic; and you'd be traveling where few people have ventured before. After a few hours, however, you'd discover the road is little more than a handful of gravel flung along the 63rd parallel.There's only one service station, and the closest hamburger is two days away. Much the same can be said of HSM. It's an intoxicating idea in orbit around some pretty spiffy technology, but it still has a way to go before it can be your fully paved data highway to limitless storage. HSM is a dandy idea, once all the potholes are fixed.
About the authors
David Burnette (david.burnette@advanced.com) is a technical editor at Advanced Systems. Cedric Higgins (cedric.higgins@advanced.com) is manager of the Advanced Systems Test Center.
Adaptive Information Systems 26001 Pala Mission Viejo, CA 92691 800-678-6978, 714-587-9077, 714-587-1933 faxAdvanced Software Concepts 2430 Vineyard Ave., #101 Escondido, CA 92029 619-737-9544, 619-737-9432 fax info@advsoft.com
Alphatronix Inc. 4022 Stirrup Creek Dr. P.O. Box 13978 Research Triangle Park, NC 27709 800-849-2611, 919-544-0000, 919-544-4079 fax sales@alphatronix.com
Amdahl Corp. 1250 E. Arques Ave., M/S 376 Sunnyvale, CA 94088 800-233-8489, 408-746-7755, 408-992-2059 fax information@efm.amdahl.com
Artecon P.O. Box 9000 Carlsbad, CA 92018 800-872-2783, 619-931-5500, 619-931-5527 fax sales@artecon.com
Automated Network Technologies 3333 S. Bannock St., #945 Englewood, CO 80110 303-789-2506, 303-789-2438 fax sales@anthill.com
Computer Associates Int'l Inc. 1 Computer Associates Plaza Islandia, NY 11788 800-225-5224, 516-342-5224, 516-342-5734 fax
Digital Equipment Corp. 334 South St. Shrewsbury, MA 01545 800-DIGITAL
Emass Inc. 10949 E. Peakview Ave. Englewood, CO 80111 800-653-6277, 303-792-9700, 303-792-2465 fax inquiry@aap.com
EMC Corp. 171 South St. Hopkinton, MA 01748 800-222-3622, 508-435-1000
FileTek Inc. 9400 Key West Ave. Rockville, MD 20850 301-251-0600, 301-251-1990 fax ljs@filetek.com
Hewlett-Packard Co. Network and System Management Division Boeblingen, Germany Contact local HP sales office for information
Hiarc Inc. 401 Pennsylvania Pkwy., #205 Indianapolis, IN 46280 317-848-2246, 317-848-9068 fax
IBM Corp. 5600 Cottle Rd. San Jose, CA 95193 800-IBM-3333, ext. *20, 800-IBM-4FAX fax
Large Storage Configurations Inc. 4211 N. Lexington Ave., M/S ARH206 St. Paul, MN 55126 800-831-9482, 612-788-6008, 612-788-6325 fax info@lsci.com
Legent Corp. 575 Herndon Pkwy. Herndon, VA 22070 800-676-5468, 708-505-9555, 708-505-9574 fax
MTI 4905 E. La Palma Ave. Anaheim, CA 92807 800-999-9MTI
OpenVision 7133 Koll Center Pkwy. Pleasanton, CA 94566 800-223-OPEN, 510-426-6486 fax
PDC 1002 W. Ninth Ave. King of Prussia, PA 19406 800-654-4PDC, 610-265-3300, 610-265-2165 fax mktg@pa.pdc.com
QStar Technologies Inc. 600 E. Jefferson St. Rockville, MD 20832 800-568-2578, 301-762-9800, 301-762-9829 fax sales@qstar.com
Storage Technology Corp. 2270 S. 88th St. Louisville, CO 80028 800-456-7689, 303-661-4800
Vangard Technology 11211 E. Arapahoe Rd. Englewood, CO 80112 800-840-6090, 303-790-6090, 308-799-9297 fax inquire@vangard.com
If you have problems with this magazine, contact webmaster@sunworld.com
URL: http://www.sunworld.com/asm-05-1995/asm-05-HSMrev.html.
Last updated: 1 May 1995.