Click on our Sponsors to help Support SunWorld
Security: Pete's Wicked World by Peter Galvin

Designing secure software

A methodology for avoiding the security holes that drive you mad

April  1998
[Next story]
[Table of Contents]
Subscribe to SunWorld, it's free!

This month Peter turns a jaundiced eye toward the sorry state of software development. Specifically, what are the flaws that cause the industry to turn out program after program with security holes. What are companies like Sun doing to correct the problem? What should they be doing? The answer: Peter's own Software Development Security Design Methodology.

Also this month: In The Buglist another summary from CERT on recently reported incidents and some more patches from Sun. Plus a few words on SANS 98 and The Hawk's compilation of security-related links. (2,100 words)

Mail this
article to
a friend
All problems fall into one of two categories: Those that can and those that cannot be easily solved. For instance, some of the denial of service attacks that have surfaced recently are a result of the IP protocol's design. Short of implementing a new protocol (any volunteers?), not much can be done beyond stopgap measures that make particular attacks less effective.

Other difficult problems include network sniffing and spoofing. These result from security-related information being sent in the clear over networks. Then there is the general authentication problem. The difficulty with authentication is that the lowest common denominator is user names and passwords, and that method is generally not sufficient.

Unfortunately, solving these problems requires new hardware, new software, and user training, all of which may not be available to everyone. Over the longer term, protocols like IPV6 and IPsec will resolve many of these issues. Of course they may create new ones. Let's leave the difficult problems aside, because they're, well, difficult.

The solvable problems are the result of poor planning, programming, and implementation. These can be solved by software vendors who spend the time and engineering effort needed to improve their coding methodologies. If we fixed these solvable problems, we would all have less work to do.

Picture this world of the future.


Is code getting better?
You could assume that the security holes in operating systems are the result of poor coding way back when, and that new code and coding methods do not have the same problem. You would be wrong. Consider Windows NT and its sorry security state. Or look in our own back yard at Solaris. Bugs in admintool, NIS+, the volume manager, procfs, PPP, PAM (the pluggable authentication module), and the PCI bus drivers (no, I have nothing against bus drivers per se, just the ones that drive the PCI buses) prove the point.

Recently released code has security holes. What about code that is currently in development? Can we count on improved quality and a new level of code integrity? In a word, no.

A friend within Sun Microsystems Computer Company's engineering group seems as frustrated by the current state of affairs as the rest of us. I quote:

The best guideline we could have would be: No idiots are allowed to write anything like setuid. More likely but still not in place now would be a policy where setuid and setgid couldn't be put back to the source base without being additionally reviewed by some panel of experts.

Unfortunately, neither of these policies are currently in place within Sun. There are groups within the software development teams who do try to apply some methods to their code creation and modification. However, there is no official Sun methodology in place to try to reduce the number of security holes released upon the world.

This should not be taken as a damnation of Sun. I'm sure that many of the other operating system vendors suffer from the same lack of internal controls. The question is, what can be done to improve the situation? Money talks, so buyers need to start speaking up and demanding higher quality code. If Sun and other OS vendors had buyers demanding that they use an improved methodology, they might start using one. That's the motivation behind SDSDM (Software Development Security Design Methodology).

SDSDM is not rocket science. If this simple methodology were applied to all appropriate programs, the world would be a better place. What are the appropriate programs? All setuid and setgid programs, and all daemons that accept network connections.

Why that set? In the first case, these programs allow users to change their access rights or to increase their privileges. Security holes therein allow users to increase their access to the system. setuid root programs allow users to gain root privileges, which is the worst exploit. In the second case, users are allowed to access the system without first getting authenticated. A network daemon may answer a network request and process it under the daemon's privileges, not a user's. Therefore, this is another way for users to increase access, or even gain initial access, to the target system.

Of course the kernel is a different case. If the kernel has security holes, no amount of checking of system programs is going to make the system secure from attack. However, relatively few kernel bugs are being found and exploited these days. Operating systems like Solaris have solid kernels that have slow-changing core facilities. Also, only the best engineers work on kernel code, reducing the chance for novice errors.

Finally, in terms of security, kernels are relatively bug-free because of the limited interfaces available to attack. For instance, Solaris 2.5.1 only has 212 system calls (check /usr/include/sys/syscall.h). Compare that to the thousands of points a hacker has available to attack: sockets, files, devices, and programs. Clearly, it's easier to secure the kernel than the remainder of the system.


The methodology
Here is SDSDM. If you have additional suggestions for SDSDM, please send them along.

Design the software with security in mind:

Implement the software following good programming practice and secure software guidelines. Appropriate information on which programming techniques, system calls, and library calls to use and avoid is not readily available. The best I located is included in the book Practical Unix and Internet Security by Simson Garfinkle and Gene Spafford. Some of the information is abstracted here, but any programmer doing security-related work would be well-advised to read this book, or at least chapter 23. Here are the dos and don'ts.



Test the software using the same methods that crackers do:

There are many more programming-level tips for networked and setuid applications available in the Garfinkle and Spafford book. Implementation of these steps would not only improve the software, it would decrease the number of security holes found, the number of patches released, and the amount of work by system administrators. It would also increase the reputation of the vendor.

The Buglist
CERT has released another summary to remind us that systems continue to be the victims of break-ins due to the failure of administrators to install patches. CERT Summary CS-98.03 reveals that weak systems are being cracked and then used to gain access to more secure machines. Crackers are installing Trojan-horse programs in place of standard system tools. These replacements are used to capture user names, passwords, and hostnames of machines that users access. The CERT summary also points to information on holes being exploited, detecting these break-ins, and recovering from them.

Sun has released a patch for the vulnerability in the volrmmount program on Solaris 2.6. The volrmmount program is a suid root program that allows users to simulate an insertion or ejection of media. The vulnerability allows a cracker to view any file on the system. For more information see Sun Bulletin #0162.

Also from Sun are patches for a vulnerability in the vacation program for all recent Solaris and SunOS versions. The bug allows attackers to access the account for which vacation is running. More information is available in Sun Bulletin #0163.

Fitting neatly into this month's column theme, Sun announced patches for a vulnerability in the dtaction program for all recent Solaris releases. The vulnerability involves incorrect bounds checking on input arguments and can be locally exploited to gain root access. Details are in Sun Bulletin #0164.

Finally, if you do nothing else for security on your 2.5.1 systems, consider installing the new version of the 2.5.1 jumbo kernel patch (103640-18). It addresses several security issues (and over 180 bugs).

Wired magazine reports that the Israeli hacker "Analyzer" tutored two California teenagers who allegedly broke into several unclassified U.S. military computers. Analyzer claims to have access to as many as 400 similar computers. Analyzer, by the way, was arrested in Israel last month.

SANS 98 (System Administration Networking and Security) is shaping up to be the best Unix security event of the year, or at least until its sister conference NS 98 rolls into Orlando in October. This year this is also an NT SANS conference, running concurrently, that should provide high-quality tutorials and talks about system administration and security in the Windows NT space. SANS 98 starts May 9 in Monterey, California. Hope to see you there.

In June, Usenix continues its string of strong conferences with its 1998 annual conference in New Orleans, LA. Details can be found at the Usenix Web page.

The Bookstore
If you are still looking to add to your online security reference pointers, you should check out "The Hawk's security links". The Hawk has put together quite an extensive set of links to security-related information.

Click on our Sponsors to help Support SunWorld


Books and articles Events Security Links SunWorld security resources

About the author
[Peter Galvin's photo] Peter Galvin is chief technologist for Corporate Technologies Inc., a systems integrator and VAR. He is also adjunct system planner for the Computer Science Department at Brown University, and has been program chair for the past four SUG/SunWorld conferences. As a consultant and trainer, he has given talks and tutorials worldwide on the topics of system administration and security. He has written articles for Byte and Advanced Systems (SunWorld) magazines, and the Superuser newsletter. Peter is co-author of the best-selling Operating Systems Concepts textbook. Reach Peter at

What did you think of this article?
-Very worth reading
-Worth reading
-Not worth reading
-Too long
-Just right
-Too short
-Too technical
-Just right
-Not technical enough

[Table of Contents]
Subscribe to SunWorld, it's free!
[Next story]
Sun's Site

[(c) Copyright  Web Publishing Inc., and IDG Communication company]

If you have technical problems with this magazine, contact

Last modified: