Securing IP The IPSec protocol suite is expected to transform the Internet, but what does this technology actually do? By Ken Masica

June  1998

For the last few years, the Internet Engineering Task Force (IETF) has been working on a suite of protocols, collectively called IPSec (IP Security), that are designed to enable secure IP connections over the Internet.

Ken Masica provides an overview of the massive amount of work being done in this area. He covers all of IPSec's major features, including the Encapsulating Security, Key Management, Security Payload, and Authentication Header protocols. He also explains how IPSec works with authentication algorithms, like HMAC-MD5 and DES-CBC. (3,100 words)

Mail this article to a friend

y now, you've probably at least heard of IPSec, the IP Security protocol. This interoperable, global security standard for the Internet has been eagerly anticipated, as it is expected to fill a major void in the current operation of the Internet. Network administrators will welcome IP Security's ability to reduce the threat of Internet attacks based on IP spoofing. Users and businesses should begin to trust the Internet more once this important standard is finalized and a security infrastructure is implemented.

IPSec promises to unleash the full potential of electronic commerce, virtual private networks, extranets among corporate partners, and private communication between individual users. Beyond these general promises, though, what is IPSec, and what does it really do?

IPSec overview
It's important to realize that IP Security (IPSec) is actually a suite of protocols rather than just a single protocol. A very large set of detailed standards documents has accumulated since work on IPSec began in 1995.

A quick tour of the Internet Engineering Task Force (IETF) IPSec Working Group home page reveals 35 Internet Draft documents (IDs) and seven Request For Comments documents (RFCs). Although not all of these documents specify protocols (the actual specificiation is 11 documents), they all play a part in defining a very broad and flexible security infrastructure for the Internet.

IPSec structure
The best place to start amidst the jungle of IPSec documents is the IP Security Document Roadmap (see Resources below for links to IPSec documents). This draft attempts to categorize all of the IPSec documents listed on the working group's home page. The IPSec document structure illustrates how the IETF standardization effort is organized and also helps locate specific IPSec information. More importantly, however, the document structure mirrors the overall IPSec architecture and its various components. All documents listed on the IPSec home page can be placed into one of the seven categories of the structure shown below.

IPSec architecture

The first major piece of IPSec as shown in the diagram is the Security Architecture. The Internet Draft, Security Architecture for the Internet Protocol, describes the security mechanisms and services for IPv4 and IPv6. It provides a good overview of the IPSec protocol suite and also describes each component of the architecture. The IPSec Security Architecture consists of the three primary protocols, shown in the second level of the diagram, as well as encryption and authentication algorithms.

The Encapsulating Security Payload (ESP) provides both encryption and authentication services. Authentication Header (AH) provides datagram authentication service. Both the ESP and AH protocols have a supporting set of documents that specify particular algorithms and options.

The Internet Security Association and Key Management Protocol (ISAKMP -- note that ISAKMP/Oakley has recently been renamed the Internet Key Exchange because, according to security group chair Robert Moskowitz, "ISAKMP/Oakley resolution is a mouthful") is another major component of the IPSec architecture. ISAKMP is the generic term for the application layer service that will support the key management requirements of the IPSec protocols. The IETF designed the Oakley Key Determination Protocol to perform the ISAKMP function. This protocol establishes a security association between communicating systems -- a framework for the generation and exchange of IPSec keying material and the negotiation of IPSec parameters.

Lastly, note the central component on the diagram, the Domain of Interpretation or DOI. The DOI ties together all of the IPSec group's documents by specifying all the mandatory and optional algorithms, attributes, parameters, and identifiers for the protocols of IPSec. It can be considered the master database of all IPSec security parameters that can be referenced by systems negotiating IPSec service.

IPSec needs a specific DOI because the ISAKMP protocol was designed to support security services at all layers of the protocol stack, not just the IP layer. Therefore, each security protocol must have its own DOI or database of all relevant parameters. Presently, only IPSec's DOI is defined.

In the remainder of this article, we'll take a closer look at the authentication, encryption, key management, and DOI components of IPSec.

Authentication
The IPSec Authentication Header (AH) is a mechanism for providing strong integrity and authentication for IP datagrams. Integrity ensures that the datagram was not altered in an unexpected or malicious manner, and authentication verifies the source of the datagram (host, user, network, etc.). AH does not support any form of encryption -- used alone, it cannot protect the confidentiality of data sent over the Internet. Rather, AH is intended to improve security in the global Internet in situations where import, export, or use of encryption may be illegal or restricted by local government regulations. AH should be free of such complications. When fully implemented, it will provide a valuable security service by authenticating IP packets and reducing the frequency of attacks based on IP spoofing.

AH takes the form of a header that is placed between the standard IPv4 or IPv6 header and the next higher-layer protocol frame (such as TCP, UDP, ICMP, etc.). Implementation of AH in IPv4 is optional and has a relatively simple IP datagram format as shown below.

On the other hand, AH implementation in IPv6 is mandatory, and the datagram format is more complex. Within the IP datagram, the AH header is placed after IP headers processed at each hop and before IP headers processed only at the final destination. The diagram below illustrates the position of the AH header within an IPv6 datagram.

The Internet Assigned Numbers Authority (IANA) has assigned the protocol number 51 to AH. This means that the IP header immediately preceding the AH header must contain the value 51 in its Next_Header field (for IPv6) or its Next_Protocol field (for IPv4).

The AH protocol provides integrity and authentication by performing a message digest calculation over the entire IP datagram. A message digest is a special one-way mathematical function that creates a unique digital fingerprint of the datagram, representable by a much smaller single number. The output of the message digest algorithm is placed into the Authentication_Data field of the AH header. The complete AH header format is shown below.

The AH protocol is designed to work with many different authentication algorithms, both those that exist today and those that will be developed for use in the future. The IPSec DOI assigns a unique number to each authentication algorithm, for identification during the negotiation process between communicating systems. Presently, two authentication algorithms have been defined -- and both are considered mandatory for compliance with IPSec. The required algorithms are HMAC-MD5 and HMAC-SHA1. HMAC stands for Hashed Message Authentication Code. MD5 is the Message Digest version 5 algorithm, and SHA1 is the Secure Hash Algorithm version 1.

HMAC-MD5 is a good example of how these algorithms work.

HMAC-MD5
The Message Digest 5 algorithm (MD5) is a one-way mathematical function. Applied to a block of data, it produces a 128-bit unique representation of that data. The 128-bit output is the compressed or digested representation of the larger block of data. When used in this manner, MD5 provides only data integrity. A message digest can be calculated on a block of data (such as an IP datagram) prior to being sent and then again after the data is received. If the two computed digests are equal, then the block of data was unaltered during transmission (assuming no malicious intervention). The MD5 algorithm is described in detail in RFC 1321.

Data authenticity can be assured by the use of secret keys when calculating the message digest. The HMAC method uses the basic MD5 algorithm to compute message digests, but operates on 64-byte blocks of data instead of the entire block of data. It also uses a secret key (known only to the communicating systems) when performing the digest calculation. HMAC is described in detail in RFC 2104.

In a typical authenticated datagram exchange using HMAC-MD5, the sender will use the previously exchanged secret key to first compute a series of 16-byte MD5 digests for each 64-byte block of the datagram. The series of 16-byte digest values are concatenated into a single value, and then placed into the Authentication_Data field in the AH header. The datagram is then sent to the receiver. The receiver must also know the secret key value in order to compute the correct message digest and match it to the received message digest for authentication. If the calculated and received digest values are equal, it means the datagram was unaltered during transit, and that it was sent by the only other party with knowledge of the secret key. Note that the usefulness of this technique is based upon the assumption that only the true sender (and not an imposter) has knowledge of the shared private key.

IPSec encryption
The Encapsulating Security Protocol (ESP) header provides integrity and confidentiality for IP datagrams. Integrity ensures that the datagram was not altered in an unexpected or malicious manner, and confidentiality ensures the privacy of the data using cryptographic techniques.

For IPv6 implementations, ESP takes the form of a header that is placed after all other IP headers in the IP datagram. Shown below is the datagram format for IPv6.

Note the two types of optional IP headers shown in the diagram. The Hop-by-Hop headers are processed by intermediate systems -- such as routers -- at each hop, whereas the Destination headers are only processed by the final end system. ESP encryption must not interfere with any portion of the IP headers that are necessary for proper delivery of the packet.

The Internet Assigned Numbers Authority (IANA) has assigned the protocol number 50 to ESP. This means that the IP header immediately preceding the ESP header must contain the value 50 in its Next_Header field (for IPv6) or its Next_Protocol field (for IPv4).

The ESP protocol, like AH, is designed to work in two modes: Tunneling mode and Transport mode. The difference is in the content of the ESP_Payload portion of the IP datagram. In Tunnel mode, an entire IP datagram is encapsulated and encrypted within the ESP_Payload. When this is done, the true IP source and destination address can be hidden as the data transits the Internet. A typical use of this mode would be for host or topology hiding during a firewall-to-firewall connection over a virtual private network. In Transfer mode, by contrast, only the higher-layer protocol frame (TCP, UDP, ICMP, etc.) is placed into the encrypted ESP_Payload portion of the datagram. In this mode, source and destination IP addresses, as well as all IP header fields and options, are sent unencrypted.

The ESP protocol has a flexible design that allows it to work with different encryption algorithms (also referred to as transforms). IPSec requires that a common default algorithm, the DES-CBC transform, be used in all ESP implementations. However, two or more systems establishing an IPSec session can negotiate use of an alternative transform. The IPSec DOI document referenced at the end of this article lists optional ESP transforms. Presently, supported optional algorithms include Triple-DES, RC5, IDEA, CAST, BLOWFISH, and RC4.

How DES-CBC works
The U.S. Data Encryption Standard (DES) is a very common encryption algorithm. First published in 1977 by the U.S. government, it is used for commercial and unclassified applications. Today, all DES patents have expired, and there are free implementations available worldwide. The IPSec ESP standard requires that all ESP implementations support DES in Cipher Block Chaining mode (CBC) as the default transform. However, its use is certainly not mandatory. Users and organizations are free to use other encryption transforms, or no encryption at all. This flexibility is especially important given the different laws around the world governing local cryptography use and export.

DES-CBC, as specified for use with ESP, works by performing a mathematical function on 8-byte blocks of data that comprise either an entire IP datagram (Tunnel mode) or the next higher layer protocol frame (Transport mode). DES-CBC replaces 8-byte blocks of unencrypted data (plaintext) with 8-byte blocks of encrypted data (ciphertext). This property of the transform facilitates in-place encryption and decryption of a data stream. A random, 8-byte Initialization Vector (IV) is used to encrypt the first block of plaintext to ensure randomness between messages that may begin with the same plaintext information.

DES-CBC's fundamental encryption feature is an identical shared secret key between the communicating parties. Because of this, it is referred to as a symmetric cryptographic algorithm. The key used by the sender to encrypt the data is the only key that can be used by the receiver to decrypt the data. Thus, effective use of DES-CBC for data privacy depends on the shared key remaining a secret between the communicating parties, without disclosure or discovery by outsiders. The key length specified for DES-CBC use within ESP is 56-bits.

The ISAKMP protocol
The Internet Security Association and Key Management Protocol (ISAKMP) is based on the central concept of a security association (SA). An SA is a unidirectional security contract between communicating systems that specifies how a secure connection will be established. The SA consists of all the parameters needed to fully specify the security contract -- items such as the authentication and encryption algorithms to be used, keying material, key lengths, and key lifetimes. All SA parameters are organized into a structure called the Security Parameter Index (SPI). An SA is negotiated for each security protocol used between communicating systems -- two systems can have multiple SAs established.

The establishment of SAs between systems takes place in two phases: First, an ISAKMP SA is negotiated between the systems (such as between two ISAKMP servers). The ISAKMP SA is then used to protect traffic between the systems for the second phase of negotiation of protocol SAs. All non-ISAKMP SAs are considered to be protocol SAs (for example, an AH SA or ESP SA). The ISAKMP protocol was designed to support security at all layers of the protocol stack, not just the IP layer and its associated AH and ESP protocols. Therefore, it is possible for other protocols (SSL, TLS, OSPF, etc.) to use ISAKMP to establish their own SAs as well. Each subsequent security protocol or service used by the communicating systems will have its own SA and corresponding SPI. The IANA has assigned UDP Port 500 for use by the ISAKMP protocol.

The Oakley protocol
Establishing the first phase ISAKMP SA involves authenticated key exchange between the communicating systems, in order to protect subsequent traffic and negotiation of protocol SAs. The Oakley Key Determination Protocol is the mechanism used to accomplish this secure exchange. Oakley uses the Diffie-Hellman (D-H) key exchange algorithm -- a public-key cryptography technique that allows systems to independently generate a unique secret key based on the knowledge of each other's public key. The ISAKMP standard requires that the public keys used to establish the ISAKMP SA be authenticated using digital signatures. ISAKMP does not specify a specific signature algorithm or Certificate Authority (CA), but instead allows systems to indicate which CAs they support. Through its implementation of Oakley, ISAKMP provides a mechanism for the identification of certificate types, supporting CAs, and the exchange of certificates between systems. Once D-H public keys are exchanged and authenticated, and a secret key generated, the communicating systems can establish an IPSec SA that characterizes the subsequent use of the AH and ESP security protocols.

The IPSec DOI
The DOI for IPSec specifies all the parameters associated with the AH and ESP protocols, and assigns them unique identifiers. In effect, the DOI serves as a database of values to be referenced during the IPSec SA negotiation. For example, as discussed previously, ESP is one of the IPSec security protocols. It consists of one mandatory encryption transform and several optional transforms. The IPSec DOI will identify the ESP protocol with a Protocol_ID of 3, and assign each ESP encryption transform a unique Transform_ID value between zero and ten. Each security protocol or service that uses the ISAKMP protocol will require its own DOI in order to specify all relevant parameters. For this reason, the IANA registers DOIs and assigns them unique identifiers. Presently, only the IPSec DOI exists and is therefore assigned the value of one.

Resources

• The IPSec Working Group home page http://www.ietf.org/html.charters/ipsec-charter.html

• ID IP Security Document Roadmap http://www.ietf.org/internet-drafts/draft-ietf-ipsec-doc-roadmap-02.txt
• ID Security Architecture for the Internet Protocol, updated May 1998 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-arch-sec-05.txt

• ID IP Authentication Header, updated May 1998 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-auth-header-06.txt
• RFC 2104, HMAC: Keyed-Hashing for Message Authentication ftp://ftp.isi.edu/in-notes/rfc2104.txt
• ID The Use of HMAC-MD5-96 within ESP and AH, updated February 1998 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-auth-hmac-md5-96-03.txt
• ID The Use of HMAC-SHA1-96 within ESP and AH, updated February 1998 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-auth-hmac-sha196-03.txt

• RFC 1827, IP Encapsulating Security Payload, updated August 1995 ftp://ftp.isi.edu/in-notes/rfc1827.txt
• ID The ESP DES-CBC Transform, updated July 1997 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-ciph-des-derived-01.txt

• ID Internet Security Association and Key Management Protocol (ISAKMP), updated March 1998 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-isakmp-09.txt
• ID Revised SA negotiation mode for ISAKMP/Oakley, updated November 1997 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-isakmp-SA-revised-00.txt
• ID The OAKLEY Key Determination Protocol, updated July 1997 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-oakley-02.txt
• ID The Internet IP Security Domain of Interpretation for ISAKMP, updated May 1998 http://www.ietf.org/internet-drafts/draft-ietf-ipsec-ipsec-doi-09.txt

• The Automotive Exchange Network (ANX) http://www.aiag.org/anx
• NIST IPSec Web-Based interoperability tester http://ipng17.ipng.nist.gov/ipsecdoc
• The ISCA IPSec certification program http://www.icsa.net

About the author
Ken Masica is an electrical engineer specializing in the areas of network design, simulation, and security. Reach Ken at ken.masica@sunworld.com.

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-06-1998/swol-06-ipsec.html
Last modified: