Tuning Unix for Web service Focus your attention on network performance, making sure your OS can handle the volume of connections you expect

January  1996

Why run a server if no one can visit? A few moments spent tuning your operating system's network parameters can make a big difference in how easy it is for people to connect to and use your site. (1,500 words)

Mail this article to a friend

If, three years ago, you had asked a coworker to find some specific resource on the Internet, you wouldn't have been surprised if they had taken a day or two to complete the request. Even tracking down popular software packages could take time, since it might take a while to locate the latest and greatest version or to accumulate all the patches. This wasn't a big deal, though, and one-day turnaround on most requests was considered acceptable.

Since the rise of the Web, we've seen a commensurate rise in impatience. Now, people get frustrated if they cannot locate a resource after ten or 15 minutes of surfing, and I've seen people break connections and move on to other pages if a document hasn't downloaded 30 seconds after they clicked the hypertext link.

The last thing any webmaster wants is to cause heartburn and frustration among those who visit his server. To that end, every webmaster should spend some time tuning server performance, making sure that everything is running smoothly on his end of the Web.

Learning to let go...
The most important part of web performance tuning is coming to grips with the fact that most of the problem is completely beyond your control. Look at the Web as a layered set of software, hardware, and communications media:

Client browser                                  Server process
Client OS                                       Server OS
Client hardware                                 Server hardware
Client connection       The Internet            Server connection

A request begins at the client browser, works its way down the left-hand side of the stack, travels across the Internet, and up the right-hand side. Your server processes the request and the result wends its way through the reverse path.

Obviously, you have no control over anything on the client side. The most critical element on the client side is the Internet connection, and you can be almost guaranteed that it isn't fast enough. Millions of people surf the net via a dialup connection, often at 14.4 kilobytes per second or slower. Most of these people are also complaining about throughput, latency, and big images that they can't download in a reasonable amount of time. Short of buying all your friends faster modems (or better yet, ISDN lines), you have no hope of improving anything on the client side.

The Internet itself is a potential performance problem. Although bandwidth is generally acceptable these days, individual Internet Service Providers vary widely in their capacity. Some folks are connected in through a T1 (1.544 megabits per second) or T3 (45 megabits per second) line and have gobs of bandwidth; others are putt-putting along with frame relay (56 kilobits per second) or ISDN (up to 128 kilobits per second). While you cannot control the client's piece of this puzzle, you may be able to throw more money at your side of the wire.

Realistically, your level of control increases as you go up the stack on the server side. You might be able to buy better hardware with faster disks and better network performance. You most likely can tweak your operating system to increase performance, and you can definitely adjust your server configuration to run as quickly as possible. Since these last two items are under the control of most webmasters, I'll focus on performance tuning at the OS level for the remainder of this column, and we'll visit server-level performance tuning next month.

Real OSes: a brief sidebar
Before I go any further, let's understand each other. When I say "operating system," I'm talking about Real Computers running Unix. Toy Computers running souped-up window systems don't qualify; anything that still refers to a file system using a single letter of the alphabet just doesn't cut it. If, during the course of installing your server, you had to decide whether to put your documents on C: or D:, or had to edit anything ending in .ini, we're about to have a parting of the ways. Best of luck to you, and let us know when you've gotten past that darned need-to-do-more-than-one-thing-at-a-time problems you've been having.

For you Mac users out there, I don't have much to offer. I started using a Mac when it was named Lisa and everything since then has been a disappointment. Easy to use, but that single-user, sort-of-multitasking architecture leaves a lot to be desired.

In short, I'm one of those obnoxious Unix bigots, and damned proud of it. (Good thing I'm writing for a Unix audience!)

Whipping your operating system into shape
Whew. Now that I've cleared that up, let's move on to OS tuning for web servers.

Web servers do not put a lot of CPU load on a machine, and they tend to have a fairly small memory footprint as well. As a result, you need to focus your attention on your network performance, making sure your OS can handle the volume of connections you can expect with your server.

The maximum load you can expect for your machine is dependent on your network connection and the size of your documents. For example, if you are running on a T1 connection with documents around 1,000 bytes, your maximum HTTP requests per second is approximately 187. Your situation is probably different; Sun has a nice chart that categorizes the maximum throughput for a variety of network connections.

Once you determine the expected load, check that your machine can handle that many connections per second. Most Unix machines have a limit on the number of pending socket connections that can queue up while current connections are being processed. This queue of pending connections is called the "listen queue" and is usually fairly short. On Solaris machines, for example, this queue is, by default, only five entries deep. Thus, if six clients attempt to connect to your machine while your server is processing a connection from another client, the sixth client will be refused.

It could happen to you
You might think only heavily loaded servers will hit this limit, but in fact the limit can be reached on lightly loaded servers when a burst of connections arrive simultaneously. This kind of bursty behavior occurs all the time with browsers like Netscape, which initiate multiple parallel requests whenever possible.

Consider the typical home page, with five or six embedded images. Netscape fetches the page itself with a single request and parses the HTML. It discovers the embedded images and immediately requests all the images simultaneously. By default, most Netscape browsers will initiate four requests in parallel; the user can set the value higher or lower. Most likely, you server gets hit with four simultaneous connection requests. If someone else is doing the same thing, you might see eight simultaneous connections, and some of those connections will be dropped when the length of the listen queue is exceeded.

What To Do?
There is an easy answer, of course: make the listen queue longer, and then make sure your server takes advantage of the longer length.

Under Solaris 2.4 and later, you can change the length of this queue using the ndd command:

     /usr/sbin/ndd -set /dev/tcp tcp_conn_req_max n

where n is the new queue length. You'll need to run this command each time you reboot; you might want to add it to an appropriate startup script in /etc/init.d.

Under Solaris 2.4, the maximum value of n is 32, unless you have applied patch 101945-34, in which case the limit is 128. In Solaris 2.5, the limit is 1024.

For other variants of Unix, it might be more difficult to make this change and you might be faced with rebuilding the kernel. For most BSD-based systems, you'll need to edit /usr/include/sys/socket.h and /usr/src/sys/sys/socket.h, looking for the definition of SOMAXCONN. Increase the value of this constant, then recompile and reinstall your kernel.

Bringing the server in line
After increasing the limit in your kernel, you still need to make sure your server takes advantage of the limit.

Regardless of the server you are running, it makes a call at some point to listen() to wait for new connections. One of the parameters to listen() is the listen queue length. Check your local documentation; this is usually the second parameter to listen().

Edit your server's source code to use a higher value for this parameter, then recompile. Now you should be taking advantage of your longer listen queue.

For further information
I've covered a fairly specific operating system change that, while somewhat complicated to make, can make a big difference to your clients. Normally, you'll never know when a short listen queue is costing you connections; taking a few moments to make the queue longer might make a lot of visitors happier about your site.

Most of what I've covered here is also discussed on the Internet. Sun has a nice set of Web Performance Tuning documents that address the listen queue problem; the folks that produce the Apache http server also have a page that addresses the problem.

Next month we'll look at server-level changes you can make to speed things up. In the meantime, if you have further questions, comments, or tips, I'd love to hear them.

Resources

• Robert McGrath's Web server performance comparison
  http://www.ncsa.uiuc.edu/InformationServers/Performance/V1.4/report.html
• Simon E. Spero's "Analysis of http performance problems"
  http://sunsite.unc.edu/mdma-release/http-prob.html
• Paul E. Hoffman's "Web servers comparison version 3.3"
  http://www.proper.com/www/servers-chart.html
• "Performance Benchmark Comparison of Unix Web Servers Using API and CGI External Gateways"
  http://home.mcom.com/comprod/server_central/performance_benchmarks.html
• Robert B. Denny's "Website performance analysis"
  http://solo.dc3.com/white/wsperf.html
• Maximum throughput for a variety of network connections
  http://www.sun.com/sun-on-net/Sun.Internet.Solutions/performance/capacity_planning/max_httpops.html
• Sun Web performance tuning docs
  http://www.sun.com/sun-on-net/Sun.Internet.Solutions/performance/index.html
• Apache http server documents
  http://www.apache.org/docs/perf.html
• Other Webmaster
  /sunworldonline/common/swol-backissues-columns.html#webmaster

About the author
Chuck Musciano has been running Melmac for close to two years, serving up HTML tips and tricks to hundreds of thousands of visitors each month. He's been a beta-tester and contributor to the NCSA httpd project and speaks regularly on the Internet, World Wide Web, and related topics. His book, HTML: The Definitive Guide, is currently available from O'Reilly and Associates. Reach Chuck at melmac@sunworld.com.

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-01-1996/swol-01-webmaster.html
Last modified: