Security: Pete's Wicked World by Peter Galvin

How to detect a break-in

Why worry? Because your site may be among the 98 out of 100 sites that fail to detect break-ins when they occur. Here is an exhaustive list of methods for break-in detection.

SunWorld
July  1997
[Next story]
[Table of Contents]
[Search]
Sun's Site
Abstract
Unfortunately, detecting a break-in is much more difficult than detecting a foul in a Holyfield-Tyson fight. Studies vary in absolute terms but are relatively clear: only two to five percent of sites detect a break-in. This month, we present an exhaustive list of methods you can use to detect a break-in. We also describe ways in which each detection method can be circumvented. It is important to realize what detection limitations there are as well.

Also in Pete's Wicked World this month: YABOB (Yet Another Buffer Overflow Bug) strikes Solaris in the buglist -- and a list of interesting upcoming security conferences.

There are several sources of information regarding break-ins and their detection. Some good ones include Computer Crime and Practical Unix & Internet Security. Unfortunately, there has not been one source of information on how to detect intrusions on Solaris systems. This month's column tries to rectify that situation. The information here will be added to the Solaris Security FAQ after you've had a chance to comment on it and send in your corrections and enhancements. (2,000 words)


Mail this
article to
a friend
You might be performing a break-in check as part of standard security maintenance procedures. If so, congratulations, because either you or your management is forward-looking and realizes that proactive security activities are one of the best insurance policies. Alternatively, you may be performing the check because you suspect you have been broken into and want to make a thorough determination. If this is your case, be sure to stop your exploration and perform a full, level-zero backup first.

The backup is necessary for two reasons. If you are interested in prosecuting whomever is tampering with your systems, then an accurate picture of the system at the time of problem detection is essential. Also, as you explore the system you can affect its state (applying the Heisenberg Principle to computer security), which can make accurate assessment of the system's security difficult.

What follows is a list of the system areas to explore when trying to detect a break-in. Here, we try to avoid duplicating information that's already in the Security FAQ. However, where the FAQ information is directly applicable and needed for completeness, it is included.

Advertisements

6) Break-in detection

6.1) Caveats

While snooping around the system, avoid leaping to conclusions. Try to perform all the main tests and roughly examine all aspects of the system. Only then should you start drawing conclusions and using the fine-toothed comb in likely areas.

Remember that there are many ways for a cracker to compromise your site's security, with the main non-technical means being physical penetration and social engineering. Evaluate these potentials as well as the operating system and applications.

Also remember that the cracker could be anyone. If you find crack tools in a trusted employee's directory, then you must consider the possibility that you've found the cause of your problems. In cases such as this, the next step could be setting traps for the suspects. For instance, replace these tools with instrumented sets that will trace what they are being used for and alert you during their use. This is the type of evidence you will need when confronting a company employee.

If there is evidence of a break-in, you need to determine a proper course of action. Do you "clean" the system and then further secure it? Do you try to detect and trap the cracker? These decision can involve corporate policies that may dictate security incident response. A lot will depend on the role of the machine in question. For instance, if the machine is not a production server, or does not contain sensitive information, then there may be leeway for long-term break-in detection instrumentation.

While exploring your site, remember that crackers can try to gain access as legitimate users and then as root, or directly as root. Protecting user accounts can be effective at keeping crackers off the systems and preventing them from growing their access rights.

6.2) Other resources
The CERT intruder detection checklist is an invaluable guide to system evaluation. It could be used in conjunction with the information here for a thorough system examination.

6.3) Determine current system state
Check current logins and watch for suspicious activity, including long logins and logins at odd times. Other signs that a cracker is using your system include uncharacteristic tool usage (compiler use by an administrative assistant), heavy system use, dial-out modem use, and odd network connections. If the cracker is currently on the system, your activities could be monitored and detection prevented. Depending on what you determine below, the tools you normally use to detect problems may have been compromised and may not be reliable. It would be best to transfer a known good copy of the programs that you use to the system under suspicion and only use those tools. Useful tools include ps, ifconfig, find, df, and ls.

Other interesting system state information includes what systems have nfs-mounted your disks. Solaris provides the dfmounts command to show current remote mounts. For instance, this output is problematic because l0pht is a known cracker site: 

# dfmounts
RESOURCE     SERVER PATHNAME    CLIENTS
  -           server /cdrom/    alliant.l0pht.com

Also, check the network status of your systems. Perhaps run snoop to watch network traffic and try to detect unexpected connections or odd packets. netstat -a shows all network daemons running on the machine. Unfortunately, general-purpose servers provide so many facilities that it is difficult to separate the wheat from the chaff via netstat.

Determine if your machine is in "promiscuous" mode via the ifstatus program.

Other system state is recorded in the system's configuration files. Careful examination of these files, including their access rights and the last modification date, can be revealing. Files to examine include /etc/inetd.conf, /.rhosts, /etc/hosts.equiv, /etc/passwd, /etc/group, /home/*/.rhosts, /etc/dfs/dfstab, and /etc/ftpusers.

Check the e-mail of root and other system directories. Some tools automatically send mail there in case of problems, and those problems can be a sign of an attack or break-in.

Scan the system for problem files. These files can include programs that are set-uid or set-gid and should not be. They can also include "trick" file and directory names such as "...", ".. ", and names that include odd characters. A trustworthy find is the best bet for these scans. Bad directory permissions, such as group or world-writable /etc can give away the shop, so all system directory permissions should be checked.

Check the cron and at queues to make sure that all your work is not being undone by automated scripts that re-open security holes.

6.4) Examine system logs
Logs can be useful not only for detecting penetration but also for determining the scope and timeframe of the break-in. However, logs can be (and are) modified by crackers to hide their tracks, so a lack of activity in the logs is not an indication that the system is secure. Useful Solaris logs include:

If syslog is redirecting logs to another host, be sure to check that host's logs as well.

The logs can indicate events that occur normally but can also be a sign of problems. For instance, a system reboot can be a bad sign. Errors messages from daemons like sendmail and finger can be caused by break-in attempts. Crackers can change the system clock to hide their trails.

6.5) Patch failure
Are the operating system patches up-to-date? If not, determine which patches should have been applied and how the associated security hole could have been used. For instance, if sendmail is not up-to-date, then check the cracker sites for exploit scripts. By examining the scripts, you may be able to find tell-tales that would be left behind if that script had been used against your site.

6.6) Binary modification
Once crackers penetrate a system, they can leave tools to help gather information from your site, or tools to allow them to regain access to the machine. Searching for these tools can be time consuming, but generally worth the effort. Several specific tools are described below, but a useful approach is to execute ls -ltr in important directories. The last files listed are the most recently changed. Either they were changed by administrators at your site (via installations or patches) or they were changed by a cracker. Typical system binaries that crackers attempt to replace include ps, ifconfig, find, login, df, and ls.

6.7) Enlist the users
Users can be either friend or foe. They can ignore your password selection recommendations, give account access to friends, login remotely with clear-text connections, and generally make the life of a security administrator a living hell. Or your users can help secure your systems. User education is one of the best tools in your arsenal. Teach users proper security methodology, and why it is important to secure the machines. If they do not feel the need for security, they will not help provide it. Users are your first and last defense against social engineering attacks: they can either grant a pushy cracker access, or they can alert you to the attack. Users can help keep their accounts secure by understanding permissions and paying attention to "last login" date information. Really helpful users can examine their last output to verify that no one else is using their accounts.

6.8) What about the applications?
Many security holes are introduced by application programs, such as Web and database servers. These applications need to be checked for configuration problems, interesting event logs, and access rights. Also, patches should be installed -- the lack of up-to-date patches can give you a place to start in your cracker hunt.

6.9) Use the Web, Luke
Because crackers use the Web to communicate vulnerability information and to brag about system conquests, it can also be used by security administrators and auditors to check for this information. For instance, search on the known cracker sites, or the Internet as a whole, for your site and machine names and the words "crack break secure" and "hack." The results can be alarming.

Useful cracker sites include:

Bug of the Month Club

In the past month Sun has released a variety of security patches. Some fix the usual buffer overflow problems, but some are more worrisome. For instance, there's now a patch to Adminsuite 2.1 and 2.2 to stop it from allowing any NIS+ user from becoming root. These are relatively new facilities, and both were written with security in mind. But still there are security holes. Likewise, there's a patch for the chkey program, which is used to change NIS+ secure keys. Unfortunately it can also be used to gain root access by overflowing its argument buffer. YABOB (Yet Another Buffer Overflow Bug) allows users of at to become root. Finally, there's YABOB in the eeprom program that allows users to become "bin," which could lead to root access. Patches are available from Sun for all of these bugs, as well as for an rpcbind bug, for most flavors of Solaris and SunOS.

Conferences

Fortunately, most companies provide travel money for security professionals to attend conferences. Unfortunately, there's usually a limit on the number of trips per year. This is especially a problem this fall, when there are several excellent conferences that are worth attending. Start buttering up the boss to let you go to:

Resources

[Peter Galvin's photo] Peter Galvin is chief technologist for Corporate Technologies, Inc., a systems integrator and VAR. He is also adjunct system planner for the Computer Science Department at Brown University, a member of the board of directors of the Sun User Group, and has been program chair for the past four SUG/SunWorld conferences. As a consultant and trainer, he has given talks and tutorials world-wide on the topics of system administration and security. He has written articles for Byte and Advanced Systems (SunWorld) magazines, and the Superuser newsletter. Peter is coauthor of the best-selling Operating Systems Concepts textbook. peter.galvin@sunworld.com

What did you think of this article?
-Very worth reading
-Worth reading
-Not worth reading
 -Too long
-Just right
-Too short
 -Too technical
-Just right
-Not technical enough
 
 
 
    

SunWorld
[Table of Contents]
Sun's Site 		[Search]
Feedback 		[Next story]
Sun's Site

[(c) Copyright Web Publishing Inc., and IDG Communication company]

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-07-1997/swol-07-security.html
Last modified: